Grounding VLM reasoning directly to visual observations and action consequences—rather than letting models generate free-form explanations—significantly improves physical reasoning generalization and reduces hallucination.
This paper tackles a key problem in vision-language models: they hallucinate reasoning that contradicts physics and misalign their explanations with actual actions. VAORA introduces two reward signals that anchor model reasoning to visual evidence and action outcomes, helping VLMs learn grounded physical reasoning that generalizes to new tasks and environments.