By targeting individual neurons rather than whole layers, you can make RAG systems more robust to noisy retrieval results—a practical improvement for real-world applications where search results aren't always relevant.
This paper proposes Neuro-RIT, a method to make retrieval-augmented language models more robust to irrelevant or noisy retrieved information. Instead of updating entire layers, it identifies and deactivates specific neurons responsible for processing bad context while optimizing others to extract useful information, improving performance on question-answering tasks.