Diffusion LLMs can generate much faster by strategically choosing which tokens to commit to first—prioritizing tokens that unlock predictions for other positions, not just the easiest ones to predict.
This paper introduces dSpark, a method for speeding up diffusion language models by learning which tokens to reveal first during generation. Instead of just predicting easy tokens, the approach identifies tokens whose revelation helps other positions become easier to predict, reducing the total decoding rounds needed.