When adapting multilingual speech systems to new languages, simple fine-tuning causes catastrophic forgetting of old languages—but selective merging strategies like SSU-Wanda+TIES can preserve both old and new language performance simultaneously.
This paper addresses expanding multilingual speech recognition systems to new languages while preserving performance on existing languages, without replaying original training data. The researchers test different adaptation and merging strategies for a connector module in speech-based LLMs, finding that careful update merging balances learning new languages with retaining old ones.