Instruction tuning unlocks zero-shot generalization — training on diverse tasks with natural language instructions teaches models to follow novel instructions.
Demonstrates that instruction-tuning a language model on many tasks described via natural language instructions substantially improves zero-shot performance on unseen tasks. FLAN outperforms GPT-3 on many benchmarks despite being 8x smaller.