You can make distillation more efficient by learning which teacher actions actually improve student performance, rather than blindly copying everything the teacher does—and you can estimate this using only the data you already collect.
This paper proposes Advantage-Based On-Policy Distillation (AOPD), which improves how students learn from teachers by selectively following teacher actions based on whether they actually help the student succeed.