When building LLMs for long-context reasoning, explicitly training models to focus on relevant evidence and avoid copying irrelevant text significantly improves both accuracy and efficiency.
This paper identifies a critical problem where large language models waste effort copying irrelevant text from long contexts instead of focusing on solving problems. The authors propose GEAR, a training method that rewards models for grounding their reasoning in key evidence while penalizing copying from distractors.