Externalizing factual knowledge to a continuous vector-indexed database lets smaller models achieve better factual accuracy and knowledge control than larger models, while keeping retrieved facts attributable and editable.
This paper introduces CO-LMLM, a language model that stores factual knowledge in an external database with continuous vector keys instead of memorizing facts in its weights. During generation, the model queries this knowledge base flexibly and retrieves human-readable information to cite.