When training smaller models to reason, letting a teacher briefly take over at critical failure points—rather than forcing the student to learn from its own mistakes—significantly improves both efficiency and accuracy.
This paper addresses a key failure mode in on-policy distillation where students get stuck in wrong reasoning paths. The authors propose Relay-OPD, which detects when a student diverges from the teacher's approach and temporarily hands off to the teacher to correct course, then resumes student training.