Standard accuracy metrics mask real behavioral divergence in quantized models—you need decision-level metrics to catch when quantized and base models disagree, even when both maintain similar overall performance.
This paper reveals that quantization (compressing LLMs to lower bit-widths) preserves accuracy metrics like perplexity but causes hidden behavioral changes. The authors introduce a new metric called correctness agreement to detect when quantized models make different predictions than base models, and analyze how quantization distorts attention weights differently across model layers.