Instead of asking LLMs to regenerate entire responses when they make mistakes, you can surgically fix just the wrong reasoning steps and use that correction to guide the model forward—saving tokens and improving accuracy.
This paper introduces Deep Interaction, a method that lets users directly edit and correct specific errors in a language model's reasoning steps rather than forcing complete regeneration. The corrected reasoning is then distilled into a prompt that guides the model along the fixed path, improving correction success rates by 25% while reducing token usage by 40%.