DARTree achieves up to 9.73× speedup on language model inference by constructing and scoring candidate token trees in parallel, accepting nearly 13 tokens per verification step—nearly 3× more than prior methods.
DARTree speeds up language model inference by using a tree-based speculative decoding approach. Instead of generating one draft token at a time, it predicts multiple token candidates in parallel using diffusion, then verifies them together. The method uses an autoregressive correction head to score entire branches at once, accepting more tokens per verification round than existing methods.