How Surprising Is Historical Italian to Language Models? Tokenization Tax, Comprehension Tax, and a Simple Mitigation

Maria Levchenko|June 25, 2026arXiv

Key Takeaway

Historical text imposes a consistent encoding penalty on LLMs, but models retain semantic understanding—making them safe for retrieval tasks if generative applications are adapted with temporal context.

Summary

This paper diagnoses why language models struggle with historical text by breaking down the problem into four dimensions: tokenization cost, predictive uncertainty, semantic robustness, and context sensitivity.

evaluation data applications

Key Terms

surprisal tokenization semantic-robustness embedding-similarity temporal-context-prompt