Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

Mehmet Iscan|July 14, 2026arXiv

Key Takeaway

Small frozen code models show no confirmed ability to operationally use error information for self-repair; what appears to work may be pattern matching rather than genuine error understanding, with important implications for deploying self-correcting code systems locally.

Summary

This paper introduces PoPE, a rigorous methodology for testing whether frozen small code models (0.5-1.5B parameters) can actually use error information to fix broken code, or if they're just responding to surface patterns.

evaluation training safety

Key Terms

code-reasoning self-critique error-correction placebo-controlled adapter-based-architecture