Late-interaction retrieval models generalize better across languages than dense models when trained on translated data, suggesting token-level matching is more effective for multilingual transfer than vector-level matching.
This paper presents open-source recipes for training retrieval models at scale. The authors curate 665M English training pairs and create two 149M-parameter models (DenseOn for dense retrieval, LateOn for late-interaction matching) that achieve state-of-the-art results.