You can now generate video and 3D animation together for any character rig—not just humans—and automatically place that animation in a 3D scene with correct camera positioning, enabling practical animation workflows from text or video.
This paper presents a model that generates both realistic video and replayable 3D motion simultaneously for any character rig. By keeping 3D motion in a separate model connected to a video generator through cross-attention layers, it avoids adding billions of parameters while supporting arbitrary skeletons.