Forgotten knowledge in continual learning isn't destroyed—it remains in a compact geometric structure. Recovery depends on how well the network's current representation aligns with past knowledge, not on information loss.
This paper investigates why neural networks can recover forgotten knowledge during continual learning. Using geometric analysis on image classification tasks, researchers found that forgotten information stays in a stable, low-dimensional structure even as the network's representations change dramatically.