Parallel decoding via discrete diffusion is viable for speech recognition and can match autoregressive performance while being faster and more efficient, using only 0.16% trainable parameters on a frozen backbone.
This paper shows that discrete diffusion language models can transcribe speech by refining entire transcripts in parallel rather than generating one token at a time. The authors adapt DiffusionGemma (a 26B model) for speech by freezing most weights and adding a lightweight audio interface, achieving competitive accuracy while transcribing in just eight parallel steps regardless of speech length.