Pythia 12B is a research-oriented model built for transparency and reproducibility rather than peak performance. It's part of a carefully documented suite of models trained on the Pile, making it unusually well-suited for studying how language models develop capabilities across scale. Expect solid general language abilities with honest limitations — it won't match more recent instruction-tuned models in following directions or chat tasks.
| Benchmark | Score | Type | Recorded |
|---|---|---|---|
| IFEval | 24.7 | accuracy | 4d ago |