Recurrent attention layers like Gated DeltaNet are actually easier to quantize than softmax attention because their state-update mechanism naturally forgets noise; you can safely quantize everything to 4-bit if you use block scaling and calibrate per-module.
This paper shows that Gated DeltaNet layers in hybrid LLMs can be quantized to 4-bit precision without quality loss, contrary to conventional wisdom. The authors quantize all 496 linear layers of Qwen 3.8-27B to NVFP4 W4A4 and demonstrate it matches full precision across benchmarks while being 17.5GB smaller and 14-19% faster.