LLMs don't memorize facts in a surface-invariant way; their ability to answer factual questions depends heavily on which name or spelling variant you use for an entity, suggesting memorization is tied to specific linguistic forms encountered during training.
This paper investigates how large language models memorize facts by testing whether they can answer questions about the same entity using different names and spellings.