You can distill LLMs without amplifying teacher bias by coupling teacher calibration with student training—using only source-domain feedback to iteratively correct the teacher while updating the student, achieving provable convergence without target rewards.
This paper addresses a key problem in LLM distillation: when you train a smaller model to mimic a larger one, you also copy the teacher's mistakes and biases. The authors propose Coupled Calibration and Learning (CCL), which alternates between correcting the teacher's errors using source-domain feedback and training the student on target questions.