Abstract :

Modern deep learning has achieved remarkable performance mainly through training very large architectures on huge volumes of data. However, this parameter-heavy paradigm is increasingly unsustainable and remains prone to overfitting whenever data is scarce. At its core, deep learning can be understood as representation learning: transforming high-dimensional inputs into structured, meaningful feature spaces. When data is limited, finding an effective representation in an unconstrained search space becomes exceptionally difficult. In this talk, we examine how inductive bias serves as a principled mathematical mechanism to guide this learning process. We will present some examples and discuss how embedding these priors into model architectures improves sample efficiency, reduces parameter counts, and helps prevent overfitting, offering a more sustainable path for model design.