Fine-Tuning Regimes Define Distinct Continual Learning Problems

Paul-Tiberiu Iordache, Elena Burceanu|April 23, 2026arXiv

Key Takeaway

When comparing continual learning methods, the choice of which model layers to train is as important as the method itself—different fine-tuning regimes can completely change which approach performs best.

Summary

This paper shows that how you choose which parts of a model to update during continual learning (learning new tasks sequentially) significantly changes which methods work best.

training evaluation

Key Terms

continual-learning fine-tuning trainable-depth catastrophic-forgetting