You can now automatically detect recurring caption errors tied to specific visual patterns in MLLM outputs, helping identify and fix systematic biases in vision-language datasets without needing model access.
This paper introduces Symbal, a method to detect systematic errors in image captions generated by multimodal AI models—cases where the same type of mistake repeatedly occurs with specific visual features.