Long-horizon reasoning failures in LLMs stem from structural biases in the reasoning space itself, not just model capacity—and injecting geometric structure into the reasoning process can dramatically improve performance on hard problems.
This paper addresses why large language models struggle with long-horizon reasoning tasks by identifying two key problems: exploration bias (getting stuck in locally plausible but structurally weak paths) and compounding bias (small errors accumulating over many steps).