Diffusion language models can outperform autoregressive models for few-shot editing tasks by filling multiple masked regions simultaneously, guided by example demonstrations and a likelihood-ratio reward.
This paper proposes using diffusion language models for few-shot prompting tasks where you need to apply changes across multiple locations in code or text. Instead of generating left-to-right like traditional models, diffusion models can fill in multiple masked regions at once, using examples to guide what gets filled in.