Previous scaling laws overemphasized model size — Chinchilla proved that training tokens should scale proportionally with parameters, reshaping how all subsequent models were trained.
Demonstrates that most large language models are significantly undertrained. For a given compute budget, the optimal strategy allocates roughly equal scaling to model size and training data. Chinchilla (70B, 1.4T tokens) outperforms Gopher (280B, 300B tokens).