Dynamically controlling solution prefix length during training—not just at data prep time—can more than double reasoning model accuracy on hard problems by keeping success rates in the optimal gradient zone, then fully removing scaffolding at test time.
This paper addresses a key problem in GRPO training: hard problems where no rollouts succeed produce zero gradient signal, wasting valuable frontier examples. AdaPrefix-GRPO solves this by dynamically prepending solution prefixes during training, adjusting prefix length via feedback control to maintain ~50% success rate (where gradient signal peaks), then removing assistance at deployment.