Training separate optimization experts for different failure modes, then merging them, beats joint optimization and lets enterprises consolidate their LLM fleet without sacrificing quality.
A company built a single self-hosted LLM to replace 200+ fragmented models by analyzing production errors and training specialized experts for instruction-following, function-calling, and task distribution. They merged these experts and achieved better performance than a 7× larger model while handling 116M monthly requests at lower cost.