Puro-2B: Poor Lab's Qwen2-1.5B Trained on RTX 5090 within $5090

Kairong Luo, Jiarui Cui, Yaorui Yin, Shengqi Chen, Yiming Yang et al.|August 27, 2026arXiv

Key Takeaway

You can now train competitive small language models on a budget using consumer hardware and low-precision training—the authors prove it's possible for under $7K and share everything needed to reproduce it.

Summary

This paper demonstrates how to train a capable 1.5B language model from scratch for under $7K using consumer GPUs and FP8 precision, making LLM pretraining accessible to resource-constrained teams. The authors release their full training recipe, data, and models openly, and derive a cost scaling law showing similar performance is achievable for ~$4.4K.

efficiency training scaling

Key Terms

fp8-precision low-precision-training curriculum-learning scaling-laws pretraining