Loss Landscapes in Deep Neural Networks: How Filter Normalization, Hessian Curvature, and Basin Flatness Explain Generalization
The parameter space of modern deep learning models spans millions to hundreds of billions of dimensions. In this high-dimensional space, the empirical risk objective forms a complex geometric surface known as the loss landscape. Despite the extreme non-convexity of deep neural networks, standard first-order optimization algorithms such as stochastic gradient descent (SGD) and Adam regularly converge to parameter configurations that achieve both low training error and robust test set generalizati
1 min