Simon Martin

PhD student · Inria Paris & École Normale Supérieure

I am a third year PhD student between the SIERRA project-team at INRIA Paris and the Center for Data Science at École Normale Supérieure. I am advised by Francis Bach and Giulio Biroli.

I am a former student of the ENS Ulm, where I obtained the ICFP Master's degree in physics. I also hold the Master's degree Mathématiques de l'aléatoire in probability and statistics from Université Paris-Saclay.

Research

I'm currently working on shallow neural networks and their learning dynamics. The goal of my PhD is to combine applied mathematics and statistical physics tools to understand gradient flow dynamics in high dimensions.

Publications

Silent Alignment in Two-Layer Linear Networks: A High-Dimensional Perspective

Simon Martin, Francis Bach, Giulio Biroli

arXiv Preprint, 2026

Paper Code

Summary and animation

We study gradient flow in two-layer linear networks with small initialization, in the high-dimensional regime where all dimensions diverge proportionally. Training happens in two phases: the network first aligns its singular directions with those of the data, while the loss stays at its initial value. Its singular values are learned only later, from the largest to the smallest.

Top: singular-value density of the network for three initialization scales, which leaves zero later when the initialization is smaller. Bottom: alignment error and loss of gradient descent; the alignment error decreases early for all three scales, while the loss waits. — Three initialization scales ρ: the alignment error drops at the same time, while the loss and the singular values wait longer when ρ is smaller. Curves show the theory, histograms show gradient descent, and the dashed line is the target spectrum.

High-Dimensional Analysis of Gradient Flow for Extensive-Width Quadratic Neural Networks

Simon Martin, Giulio Biroli, Francis Bach

JMLR Journal of Machine Learning Research, 2026

Paper Code

Summary and figure

We study gradient flow for a two-layer quadratic network trained on a finite dataset, with a width proportional to the dimension. In this limit, the dynamics behave like training on infinitely many samples, with an additional noise term and a memory term. With weight decay and above a width that we compute, gradient flow finds the same solution as a network of unlimited width. With label noise, the test error shows a double descent in the number of samples.

Left: test error against the number of samples, with a peak at the interpolation threshold for weak weight decay. Right: smallest width that reaches the same solution as a network of unlimited width, for three values of the label noise variance. — Left: test error as the number of samples grows, with label noise. Right: smallest width that reaches the same solution as a network of unlimited width.

Bayes-optimal learning of an extensive-width neural network from quadratically many samples

Antoine Maillard, Emanuele Troiani, Simon Martin, Lenka Zdeborová, Florent Krzakala

NeurIPS Advances in Neural Information Processing Systems, 2024

Paper Code

Summary and figure

We study how many samples are needed to learn a two-layer network with quadratic activation, whose width is proportional to the input dimension. We compute the best test error that any method can achieve from quadratically many samples, and we provide an algorithm based on AMP that reaches it. Empirically, gradient descent reaches twice this error, while averaging over random initializations recovers the optimum.

Best achievable test error against the number of samples, without label noise, for four normalized widths: each curve decreases from 1 to 0, reached at the perfect-recovery threshold. — Best test error that any method can achieve without label noise, as the number of samples grows, for four widths. It reaches zero at the perfect-recovery threshold.

On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensions

Simon Martin, Francis Bach, Giulio Biroli

AISTATS International Conference on Artificial Intelligence and Statistics, 2024

Paper Code

Summary and figure

We study gradient flow for a two-layer network with quadratic activation learning a teacher network of the same type, with infinite data. Perfect recovery occurs as soon as the network has as many neurons as the teacher. Additional neurons can however slow down convergence, from exponential to polynomial, as the unnecessary directions vanish slowly.

Overlap between the learned network and the teacher as a function of the student and teacher widths: partial recovery above the diagonal, full recovery below it. — Overlap with the teacher at the end of training, as a function of the student and teacher widths.