High-quality code documentation doesn't improve AI agents' ability to fix real bugs, suggesting that documentation quality and real-world problem-solving are decoupled—a finding that challenges assumptions about documentation's utility for coding agents.
This paper investigates whether better code documentation helps AI agents fix bugs in real repositories. The authors create a benchmark to measure documentation quality (based on whether code can be regenerated from descriptions) and an optimizer to improve it.