A single small embedding model can serve multiple low-resource languages effectively through targeted fine-tuning with synthetic data, but scaling up to larger multilingual models remains necessary for competitive performance on these languages.
Researchers fine-tuned a small 300M embedding model (EmbeddingGemma) to work across four Central Asian languages—Tajik, Kazakh, Uzbek, and Kyrgyz—using synthetic question-answer pairs generated by a language model. The fine-tuned model significantly improved retrieval performance (14-26 percentage points) on a multilingual benchmark, though it still lags behind larger 560M multilingual models.