Agentic auditing can efficiently find corpus-specific data quality issues by treating an LLM as a debugger that compiles findings into reusable tests, costing $2-3 per run versus $12k-90k for full-corpus LLM classification.
Researchers used an LLM-powered agent to audit the Slovak portion of FineWeb2, a multilingual web corpus. Instead of classifying documents, the agent explored the corpus through an API, discovered data quality issues, and compiled them into executable tests.