LLMSurgeon: Diagnosing Data Mixture of Large Language Models

Yaxin Luo, Jiacheng Cui, Xiaohan Zhao, Xinyi Shang, Jiacheng Liu et al.|May 28, 2026arXiv

Key Takeaway

You can audit an LLM's training data composition by analyzing its outputs, even without access to the original training corpus, using statistical techniques to correct for classifier confusion and recover the underlying data mixture.

Summary

This paper introduces a method to reverse-engineer what data was used to train large language models by analyzing their generated text.

data evaluation training

Key Terms

label-shift-assumption confusion-matrix inverse-problem domain-taxonomy training-data-composition