Functional test passage alone is insufficient for evaluating coding agents—real-world software development requires meeting code review constraints, and current agents have a significant gap between passing tests and satisfying these constraints.
SWE-Gate is a benchmark that evaluates coding agents not just on passing tests, but on meeting real-world code review standards. It includes 303 repair tasks from Python repositories with explicit review constraints derived from actual pull request comments, revealing that many patches pass functional tests but fail review requirements.