Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye et al.|August 4, 2026arXiv

Key Takeaway

Test-time scaling isn't one thing: different inference strategies have different statistical structures and failure modes. Comparing them requires protocol-matched reporting of compute, uncertainty, and the exact inference method used—not just a single 'budget' number.

Summary

This paper systematizes how LLMs can solve harder problems by spending more compute at inference time. It clarifies that different 'test-time scaling' approaches (like trying multiple solutions or searching through partial answers) work fundamentally differently, making direct comparisons misleading.

reasoning evaluation

Key Terms

test-time-scaling single-trajectory-sequential-scaling leaf-level-scaling-with-terminal-reduction prefix-level-scaling inference-protocol