Evolution Strategies offer a fundamentally different optimization approach for LLM reasoning—trading single-attempt accuracy for diversity of correct solutions—making it complementary to standard policy optimization methods rather than just a memory-efficient alternative.
This paper investigates Evolution Strategies (ES) as a post-training method for improving LLM reasoning. ES maintains a population of model variants and selects based on performance, unlike GRPO which optimizes a single model.