When building LLM agents with large tool outputs, retrieval-based access can cause accuracy loss not because agents fail to compute correctly, but because they ignore retrieved evidence and use their own outputs instead—a failure mode invisible in final accuracy metrics.
This paper investigates how LLM agents lose accuracy when tool outputs are retrieved from files rather than kept in context. Using PHREEQC, a geochemistry simulator, the authors show that GPT models suffer 7.9-11.3 point accuracy drops under retrieval, primarily because they substitute their own outputs for the authoritative tool results. Claude models show no detectable loss.