Catastrophic forgetting can be analyzed without assuming specific model architectures by tracking how sample perturbations propagate through tasks, revealing that learning-rate schedules and gradient alignment are key factors controlling how much previous knowledge is lost.
This paper develops a theoretical framework to understand catastrophic forgetting in continual learning—when learning new tasks hurts performance on old ones. Using algorithmic stability analysis, the authors decompose forgetting into measurable components and derive bounds showing how learning rates, task structure, and optimization dynamics affect memory retention across sequential tasks.