2025/2026 PSL Week on Statistical Physics and Machine Learning

The past decade has witnessed a surge in the development and adoption of machine learning algorithms to solve day-a-day computational tasks. Yet, a solid theoretical understanding of even the most basic tools used in practice is still lacking, as traditional statistical learning methods are unfit to deal with the modern regime in which the number of model parameters are of the same order as the quantity of data – a problem known as the curse of dimensionality. Curiously, this is precisely the regime studied by Physicists since the mid 19th century in the context of interacting many-particle systems. This connection, which was first established in the seminal work of Elisabeth Gardner and Bernard Derrida in the 80s, is the basis of a long and fruitful marriage between these two fields.

The goal of this PSL week is to provide an in-depth overview of these connections and a good vision of the different tools available in the statistical physics toolbox, as well as their scope and limitations.

Course Information

• Instructors: Bruno Loureiro (CNRS & DI-ENS) and Antoine Maillard (INRIA & DI-ENS)
• Dates: 11-15 April, 2022
• Location: See PSL ENT or contact us.

Evaluation

Assiduity + Paper review.

Course Description

• Historical background and introduction to statistical physics of learning.
• Statistical-to-computational gaps in high-dimensional inference: the spiked Wigner model, BBP transition, AMP, typical case complexity.
• High-dimensional theory for one-pass SGD: classical rates for convex optimization, scaling limits in high-dimensions, convergence rates for non-convex cases (the information exponent)
• Non-convex landscapes in high-dimensions: the Kac-Rice formula

Requirements

Basic probability theory, linear algebra and analysis. No background in statistical physics will be assumed.

Recommended literature

• For the BBP transition and Kac-Rice, Antoine’s lecture notes on Statistical and computational phase transitions in high-dimensional statistics
• Classical SGD theory, Chapter 5 in Francis’ book.
• For one-pass SGD, Bruno’s lecture notes.
• To go further, Bruno’s lecture notes on Four lectures on Statistical Physics of Learning.

Course Schedule