Arxiv_dimfree_2023

Our new paper titled From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks is now out! In this work, we derive different limits for the one-pass SGD dynamics of two-layer neural networks, including: a) the classical gradient flow limit; b) the high-dimensional limit studied by Savid Saad & Sara Solla in the 90s; c) and the more recent infinite width mean-field limit. Check it out!