Foundation models don't need separate training algorithms for different stages—you can train everything with one likelihood-based method by treating RL objectives as fitting to model-generated distributions, which improves efficiency and capability coverage.
This paper shows that foundation model training stages (pretraining, fine-tuning, reinforcement learning) can be unified under a single likelihood-based algorithm. Instead of switching between different training methods, the researchers reformulate RL objectives as maximum-likelihood fitting, allowing all stages to use the same core algorithm.