Language models don't store facts in a single, consistent way like traditional databases do. Instead, they encode knowledge in task-specific parameter subsets, meaning the same fact may be retrieved differently or not at all depending on how you ask the question.
This paper investigates whether language models store factual knowledge like unified databases or in task-specific ways.