The divergence between a proxy metric (learned reward) and true performance when the proxy is optimized directly.