Modality imbalance (text drowning out vision) is a real bottleneck in multimodal reasoning. By detecting when visual input is being ignored and training only on those critical tokens, you can make self-distillation much more effective.
This paper identifies and addresses modality imbalance in multimodal language models—where text dominates over visual information during reasoning. OPD-V uses positive and negative teacher models with modified images to detect when the model isn't properly using visual input, then selectively applies self-distillation only on tokens where visual information matters most.