By parameterizing the KL penalty weight (β) in self-distillation, you can trade off stability and teacher guidance more flexibly, and implement the optimal policy through simple logit mixing rather than expensive RL optimization.
This paper improves on-policy self-distillation for reasoning models by introducing β-OPSD, which treats the KL penalty weight as a tunable parameter rather than a fixed value. The method derives an optimal policy as a geometric blend between a reference policy and a teacher model, then implements this efficiently through logit mixing during distillation.