Accepted tokens per round is a misleading metric for speculative decoding speedup—you must also measure completion length and verification rounds, as relaxed verification often trades token acceptance for longer outputs with minimal real-world speed gains.
This paper reveals a critical flaw in how we measure speedup in speculative decoding with relaxed verification: while accepting more tokens per round looks faster, completions actually become 1.54× longer, offsetting efficiency gains.