Scaling up language models unlocks zero-shot task performance — GPT-2 showed that bigger models learn general capabilities, not just better next-token prediction.
Introduces GPT-2, a 1.5B parameter language model that performs tasks zero-shot by simply conditioning on natural language descriptions. Demonstrates that language models implicitly learn to perform tasks without explicit supervision as they scale.