Superposition and Sparse Autoencoders: Mathematical Foundations, the Polysemanticity Bottleneck, and Dictionary Learning in Language Models
For years, attempts to interpret transformer neural networks by inspecting individual neurons encountered an obstinate barrier: polysemanticity. A single neuron in an intermediate multi-layer perceptron (MLP) or residual stream layer rarely corresponds to a single human-interpretable concept. Instead, the same neuron frequently fires on a disparate mixture of inputs, such as Python syntax errors, discussions of Renaissance art, and Spanish verbs. This phenomenon prevents mechanistic interpretabi
1 min