RTL repair benchmarks need careful evaluation design—simulation and formal methods can disagree, and you must verify models actually use the specification, not just get lucky with fixes.
This paper creates RTLRepairBench, a dataset and evaluation framework for measuring whether AI models can fix broken hardware designs (RTL code). The authors identify critical measurement problems: simulation and formal verification give contradictory results, and models might appear to fix code without actually using the specification.