Diffusion-based language models outperform auto-regressive models for formal theorem proving by generating multiple tokens simultaneously, enabling better long-range coherence and error recovery—a paradigm shift for mathematical reasoning tasks.
This paper introduces Diffusion-Proof, the first framework using diffusion language models (which generate text by iteratively refining multiple tokens at once) for formal theorem proving. Unlike traditional auto-regressive models that predict one token at a time, diffusion models better maintain long-range coherence needed for complex proofs.