In-context learning emerges at scale — GPT-3 showed that sufficiently large language models can perform tasks from just a few examples in the prompt, without training.
Introduces GPT-3, a 175B parameter autoregressive language model that achieves strong few-shot performance on many NLP benchmarks by conditioning on a few examples in context, without any gradient updates or fine-tuning.