On-Policy Self-Distillation without Any Supervision

Yijiang Li, Bingyang Wang, Yijun Liang, Yunjie Tian, Di Fu et al.|August 6, 2026arXiv

Key Takeaway

You can improve LLMs through self-distillation using only the model's own outputs and internal consistency, without needing ground-truth labels or external feedback—making it truly self-supervised post-training.

Summary

This paper introduces Unsupervised On-Policy Self-Distillation (U-OPSD), a method that improves language models without requiring external supervision like ground-truth answers or feedback from larger models.

training reasoning efficiency

Key Terms

on-policy-learning self-distillation self-consistency majority-vote pseudo-labels