Language model performance is predictably governed by power laws in parameters, data, and compute — enabling rational decisions about how to scale models.
Discovers that language model performance follows predictable power-law relationships with model size, dataset size, and compute budget. These scaling laws hold across seven orders of magnitude and enable principled allocation of training resources.