Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology

Yusuf Salcan, Simon Ging, Robin Schirrmeister, Philipp Arnold, Elmar Kotter et al.|June 18, 2026arXiv

Key Takeaway

You can train medical vision-language models to perform spatial grounding (locating regions in images) alongside report generation without sacrificing language quality, using automatically-curated training data instead of expensive manual annotations.

Summary

This paper introduces RefRad2D, a large-scale bilingual dataset of 1.2M medical images paired with text, and RadGrounder, a vision-language model trained to simultaneously generate radiology reports, answer visual questions, and locate anatomical regions via bounding boxes or segmentation—all without manual spatial annotations.

multimodal data

Key Terms

vision-language-models spatial-grounding visual-question-answering training-data-curation automated-segmentation