Language models can navigate knowledge graphs comparably to humans but don't yet surpass them; performance varies significantly by language and domain, suggesting these models have real limitations in exploratory information retrieval tasks.
This paper evaluates language models on the Wikipedia Game—a task where players navigate from one article to another using only hyperlinks. Researchers tested embedding models and LLMs across multiple languages and academic datasets, building large local Wikipedia and citation graphs.