Diffusion models can generate multiple tokens in parallel, but you need a smart controller to decide when to stop and how much to generate; this work shows you can cut inference costs by ~20% by training a lightweight decision module on agent trajectories.
This paper introduces a metacognitive controller that optimizes how diffusion language models generate text during agent interactions. By learning when to stop generating tokens and how long responses should be, the controller reduces computational cost by 20.6% while maintaining task performance—making parallel token generation more practical for real-world agents.