Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

Teng-Ruei Chen|July 9, 2026arXiv

Key Takeaway

When you have a limited budget per query and an imperfect way to check if outputs are correct, you should dynamically decide whether to resample your current model or switch to a different one based on estimated correctness gains per unit cost.

Summary

This paper addresses how to optimally allocate a per-query budget between resampling (generating multiple outputs from the same model) and rerouting (switching to a different model) when using an imperfect verifier to check correctness.

efficiency evaluation

Key Terms

test-time-compute routing verifier best-of-k-sampling cost-quality-tradeoff