Turn-level best-of-N sampling with off-the-shelf LLMs can improve agent performance on complex tasks without retraining—just by having a selector model pick the best action at each step.
This paper tests a simple inference-time technique where one LLM generates multiple candidate actions and another LLM selects the best one at each step of a long-horizon task. Testing on complex agent tasks shows 5-10% accuracy improvements and more consistent behavior, with agents making fewer attempts and better following instructions.