Neural networks don't just get better at tasks with scale—their internal neuron populations reorganize predictably, with interpretable neurons becoming more specialized while others remain general, offering a new lens for understanding how model size shapes network structure.
This paper reveals that as neural networks grow larger, certain neurons called Rosetta Neurons become more selective and specialized, following predictable scaling patterns. While these interpretable neurons increase in absolute number, they shrink as a percentage of total neurons, and the remaining neurons become less selective—a phenomenon the authors call Neuron Polarization.