Language models can understand safety instructions but don't prioritize them during execution—adding explicit obstacle-aware planning and verification mechanisms dramatically improves both task success and collision avoidance in robot control.
This paper addresses safety in coding agents for robot manipulation by introducing SafeHarness, a system that prevents collisions with obstacles during task execution. The key insight is that language models can reason about obstacles but fail to prioritize safety constraints during planning and contact execution.