Video generators marketed as world models don't yet capture the correct distribution of possible futures—they produce plausible individual videos but fail to match the statistical likelihood of different physical outcomes when given the same starting conditions.
This paper introduces PAWBench, a benchmark that evaluates whether video generation models can accurately reproduce the distribution of possible physical outcomes, not just individual plausible videos. Current models fail to match real-world probability distributions of behaviors, revealing a significant gap in what it means to be a true world model.