Frontier-class performance for $5.6M in compute — DeepSeek-V3 demonstrated that engineering efficiency can close the gap with models costing orders of magnitude more to train.
Presents DeepSeek-V3, a 671B MoE model (37B active) trained on 14.8T tokens for $5.6M in compute. Achieves GPT-4o and Claude 3.5 Sonnet class performance while being fully open-weight, using FP8 training and multi-token prediction.