Latest news and events
We keep you up-to-date.
Latest news
There are no current news available at the moment.
Current events
-
15 OctMIP Seminar: Raphaël Berthier (INRIA Sorbonne Université)Title: The multiple timescales of gradient descent on the edge of stability
MIP Group Seminar (past events)
Data-driven regularization and infinite dimensional learning architectures
by Samira Kabri (University of Hamburg)
In recent years, numerous data-driven strategies to solve inverse problems have emerged. In this context, the inverse problem is usually formulated in a probabilistic setting as it is known from the field of Bayesian inverse problems. The reconstruction approaches themselves however, are often deterministic and more related to classical methods. In this talk, we transfer the classical concept of regularization to a typical self-supervised learning setting, in which access to ground-truth data and the forward operator is assumed. To take into account instabilities arising from an infinite dimensional problem formulation, the theoretical framework is based on infinite dimensional reconstruction operators. Therefore, we first study linear, data-driven reconstruction operators based on spectral filtering and derive conditions under which convergent data-driven regularization methods can be obtained. We then discuss possibilities and challenges to implement such reconstruction operators in practice on the basis of Fourier neural operators and their connection to convolutional neural networks.
Scaling predictive coding with μP
by El Mehdi Achour (UM6P)
Predictive coding is a promising local alternative to backpropagation, but its scalability to modern deep networks has remained unclear. This talk presents a unified perspective on scaling predictive coding networks through μP-inspired parameterisations: how the right scalings make very deep predictive coding networks trainable in practice, and why these scalings arise theoretically in the infinite-width and infinite-depth regimes. For linear networks that are much wider than they are deep, the theory shows that predictive coding recovers backpropagation-like gradients under the appropriate limits, while experiments suggest that these insights extend to nonlinear networks. Together, these results show that predictive coding can be made stable at depth, transfer hyperparameters across scales, and offer a principled path toward scalable local learning.
This talk is based on the two recent papers “MuPC: Scaling Predictive Coding to 100+ layers” https://arxiv.org/pdf/2505.13124 and “On the Infinite Width and Depth Limits of Predictive Coding Networks” https://arxiv.org/pdf/2602.07697.
Two timescale training dynamics in Deep learning
by Scott Petsme (INRIA Grenoble)
Neural networks trained with gradient methods often exhibit surprising dynamics with distinct timescales. Two notable examples are incremental learning, where features are acquired in order of increasing complexity, yielding training curves with long plateaus separated by sharp transitions, and grokking, where the training loss quickly reaches zero while generalisation is delayed. Despite being observed, these phenomena remain poorly understood. In this talk, I will present two works that explain these behaviours by characterising the full optimisation trajectory in suitable asymptotic regimes. First, for gradient flow on a two-layer diagonal linear network with vanishing initialisation, the trajectory jumps between saddle points of the loss, explaining incremental learning: coordinates activate sequentially from zero until the minimum l1-norm interpolating solution is reached. Second, for gradient flow with vanishing l2-regularization on a general non-convex loss, the trajectory first converges rapidly to a manifold of critical points, then slowly drifts along it while decreasing the l2-norm. This norm-minimisation phase explains grokking, since smaller-norm solutions often generalise better.