You can improve code generation by sampling from your model's own outputs and fine-tuning on them—no external tools needed. The gains come from balancing precision (removing bad options) with exploration (keeping useful diversity).
A simple technique called self-distillation improves code generation in large language models by having them sample their own outputs and fine-tune on those samples. The method boosts performance significantly (42.4% to 55.3% on benchmarks) without needing external verifiers or teacher models, and works across different model sizes and architectures.