Static retrieval metrics (measuring if a document answers the current question) are nearly independent from causal utility in multi-step agent search. Optimizing traditional RAG systems won't improve agentic reasoning; you need metrics that measure whether documents enable better future queries.
This paper reveals a critical gap in how retrieval systems are evaluated for AI agents. Traditional retrieval metrics measure whether a document answers the current question, but agents using multiple search steps need documents that enable better future queries.