Speculative decoding isn't just a speed trick—it changes how models reason over long horizons. By training draft models to fail at critical moments, you can force the main model to handle important reasoning steps with better precision, improving agent accuracy on complex tasks.
This paper shows that speculative decoding—a technique usually just for speed—actually affects reasoning quality in long-horizon AI agents. The authors discovered that numerical noise accumulates during the verification process, degrading reasoning accuracy over many steps.