Steering beyond Language: Improving Multilingual Training-Free Text Embeddings via Paired-Residual Subspace Removal

Ailiang Lin, Zhuoyun Li, Yusong Wang, Keyu Mao, Kotaro Funakoshi et al.|October 16, 2026arXiv

Key Takeaway

Language-specific noise in LLM embeddings can be removed at inference time using paired multilingual sentences, significantly improving cross-lingual and monolingual embedding quality without any training.

Summary

This paper tackles multilingual text embeddings from frozen language models by identifying that semantically equivalent sentences misalign across languages due to language-specific geometry.

multimodal efficiency evaluation

Key Terms

training-free multilingual-embeddings subspace-removal language-invariant cross-lingual-alignment