You can dramatically speed up identifying problematic training data by training a lightweight encoder to predict influence rankings, rather than computing expensive influence functions directly.
This paper proposes Influcoder, a method that speeds up data attribution for large language models by distilling influence function computations into a compact encoder. Instead of expensive influence calculations, it learns to quickly identify which training samples contributed to specific model outputs, enabling efficient dataset curation and quality filtering.