LLM agents can automate reproducibility checks and robustness testing in scientific research, flagging analytical choices that might invalidate findings—reducing manual review burden while improving research quality.
This paper develops AI agents to automatically reproduce scientific findings and test their robustness. Using medical studies from a cancer registry, the system successfully reproduces 77% of claims and identifies analytical choices that could affect conclusions, helping experts focus on potentially problematic research decisions.