By using imagined rollouts from a frozen world model to rehearse old tasks while learning new ones, robots can continuously improve on sequential tasks without catastrophic forgetting—even learning entirely in imagination when the world model is accurate enough.
This paper presents Dream to Remember (D2R), a method for continually improving robot policies on new tasks without forgetting earlier skills. Instead of retraining the policy, D2R uses a frozen world model to imagine practice on old tasks while a small steering module learns new ones, avoiding catastrophic forgetting that plagues standard fine-tuning approaches.