Looped models can be dramatically more efficient by recognizing that recurrent states converge to fixed points, enabling truncated backpropagation, KV cache sharing, and faster RL—with learned depth priors and orthogonal injection providing better supervision than existing approaches.
This paper optimizes looped language models—models that process information through multiple recurrent passes—by leveraging fixed points in recurrent states.