For multimodal AI systems, alignment-only training catastrophically fails on compositional reasoning when modalities carry different types of information—you need to explicitly preserve modality-specific structure alongside alignment.
This paper shows that standard cross-modal alignment alone loses important information when modalities are asymmetric (like text queries and images). It introduces Decomposed JEPA, which splits learned representations into a shared aligned part and private modality-specific parts, recovering information that alignment discards and improving performance on reasoning tasks.