Video diffusion models can be repurposed as action encoders to create embodiment-agnostic action representations that transfer across robots and humans without retraining, while being 300× more compact than raw video.
DreamX repurposes a pretrained video diffusion model to learn universal action representations that work across different robots, humans, and environments.