When building with LLMs, don't assume fancy self-improvement techniques help—repeated sampling at the same token cost is a stronger baseline, and self-inspection methods often fail to improve over it.
This paper compares self-improvement methods (like self-critique, reflection, and debate) against simple repeated sampling at equal token budgets across models from 1.5B to 7B parameters. The key finding: repeated sampling consistently matches or beats elaborate self-inspection methods, and self-inspection methods actually get worse as models scale.