Using the difference between a reasoning-tuned teacher and its base model as a distillation target is more effective than imitating the teacher's outputs directly, enabling faster and better reasoning capability transfer.
This paper proposes On-Policy Delta Distillation (OPD²), a new approach for training reasoning-focused language models. Instead of copying a teacher model's outputs directly, it uses the difference (delta) between the teacher and its pre-training version as a training signal. This captures what the teacher learned about reasoning and transfers it more effectively to student models.