When building long-context systems, irrelevant nearby text can be a bigger problem than distance itself. LYRA's t-distributed approach helps models focus on what matters regardless of position.
This paper identifies the 'Proximity Trap'—where LLMs struggle with distant evidence not because of distance itself, but due to interference from irrelevant nearby context. The authors propose LYRA, a retrieval mechanism using t-distributed matching to prioritize task-relevant evidence while maintaining positional information, and introduce ProxBench to benchmark this problem.