You can extract and cache the reasoning logic from LLM problem-solving into reusable CPU programs, eliminating redundant computation and dramatically reducing inference costs while maintaining performance.
This paper introduces agentic reasoning caching (ARC), a method that extracts reasoning logic from LLM traces and converts it into efficient, CPU-executable programs. Instead of re-solving problems from scratch, ARCs apply learned solution patterns to new instances, achieving comparable performance to frontier LLMs while being 16-291x cheaper and fully explainable.