You can build effective multi-domain systems for low-resource languages by reusing intermediate checkpoints as specialists and routing based on visual style, rather than training separate models from scratch for each domain.
This paper tackles historical Manchu text recognition across multiple visual styles (regular script, running script, palace memorials) with limited training data. The authors build a multi-expert system that reuses fine-tuned model checkpoints as domain specialists and uses a lightweight image classifier to route pages to the appropriate expert.