Pretraining data poisoning is practical at web scale through public discussion interfaces, and detecting whether poison survives data curation pipelines is critical for understanding real-world LM security risks.
This paper shows that language models can be poisoned during pretraining by injecting malicious content into public discussion forums and other web-scale sources.