2024 ICML ICML 2024

Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning