Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

Matthew L. Smith, Jonathan P. Shock, Samuel T. Segun, Iyiola E. Olatunji, Tegawendé F. Bissyandé|May 18, 2026arXiv

Key Takeaway

LLM factual accuracy isn't random—it scales predictably with model size and training data frequency, meaning you can estimate what facts a model will reliably remember based on these two factors.

Summary

This paper reveals that LLM factual recall follows a predictable pattern based on two factors: model size and how often a topic appears in training data.

scaling evaluation training

Key Terms

scaling-laws factual-recall training-distribution signal-to-noise-ratio