MEDIAREF enables reproducible, cost-effective evaluation of how well LLMs can assess source credibility for fact-checking, addressing a gap where existing systems assume all retrieved evidence is equally reliable.
This paper introduces MEDIAREF, a public database of web documents from 200 media sources designed to help AI systems verify information credibility. Instead of relying on expensive search APIs, researchers can now use MEDIAREF to test how well language models assess whether news sources are trustworthy—a key step in fact-checking systems that cite their sources.