How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias, Thaís Gaudêncio, Yuri Malheiros et al.|October 22, 2026arXiv

Key Takeaway

Retrieval evaluation scores are highly sensitive to annotation completeness—expanding ground-truth evidence annotations improved reported metrics without changing what the system actually retrieved, suggesting evaluation protocols must account for annotation coverage to avoid misleading perform...

Summary

This paper evaluates a retrieval system for a clinical AI tool that finds relevant medical evidence about breast lesions. The researchers tested different ways to chunk documents and rank results, finding that a hybrid approach combining multiple retrieval methods achieved strong performance.

evaluation applications data

Key Terms

rag-pipeline dense-embeddings cross-encoder ndcg annotation-coverage