Papers

Yaohui Zhang, Binxu Li, Haoyi Duan et al.

Current multimodal models can approximate values from scientific figures but struggle with precision; providing source data instead of figures dramatically improves accuracy (90% to 97.4%) while reducing computational cost, suggesting a practical path for scientific data extraction.

PlotGround is a benchmark for evaluating how well AI models can extract numerical values from scientific figures.

Hanchu Zhou, Dechen Gao, Hang Wang et al.

Using semantic communication between robots—where they exchange meaningful descriptions rather than sensor data—enables better coordination on long-horizon tasks while keeping each robot's execution independent and reliable.

DuoMind is a framework that enables multiple robots to coordinate and work together on complex tasks by combining vision-language models for high-level reasoning with vision-language-action models for precise execution. Robots communicate through semantic messages rather than raw data, allowing them to share understanding of the task and environment while maintaining independent control.

Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc et al.

Spatially grounded self-distillation with synthetic data can teach multimodal models better visual reasoning that transfers to real-world tasks, without requiring human annotations or external teachers.

This paper improves multimodal AI models by having them learn from a smarter version of themselves that receives spatial hints about where to look in images. Using synthetic scenes with automatic object labels, the approach trains models to understand spatial relationships without human annotation, and surprisingly, these improvements transfer to real-world vision tasks.

Yakun Zhu, Yi Bin, Yujuan Ding et al.

Separating geometry into common and residual components while routing visual learning through structured latents prevents representation collapse and improves 3D spatial reasoning from images.

This paper addresses 3D spatial reasoning from 2D images by introducing GeoLatent, which uses decomposed spatial representations (position, direction, geometry) with geometric supervision and routed optimization. The method prevents geometry collapse and ensures latents are actively used during learning, achieving state-of-the-art results on spatial reasoning benchmarks.

Tian Xia, Minghao Liu, Yiqing Liang et al.

For imbalanced clinical tasks, optimizing prompts for ranking metrics (AUROC) instead of accuracy can dramatically improve model performance—up to 16 percentage points—because accuracy-based optimization fails when one class dominates the data.

This paper addresses class imbalance in clinical diagnosis by optimizing multimodal language models for AUROC instead of accuracy. The authors introduce Ranking-PE, a prompt optimization method that evaluates candidate prompts based on how well they rank positive cases above negative cases, rather than raw correctness.

Xinghao Chen, Xiangbo Gao, Jiongze Yu et al.

Video text editing requires balancing three competing goals: correct text, smooth motion, and unchanged background. This benchmark and dataset help measure those trade-offs and establish baselines for the community.

ViTeX-Bench is a benchmark for video scene text editing—replacing text on surfaces like signs and labels while keeping the rest of the video unchanged. It includes 387 real videos, evaluation metrics for text accuracy and visual quality, and a baseline editor that achieves strong results. This addresses a gap where video text editing lags behind image editing.

Owais Iqbal, Sudipta Sarkar, Shyam Marjit et al.

You can efficiently learn video representations by repurposing pretrained image models with clever masking strategies, avoiding expensive 3D architectures and reconstruction overhead.

VideoMSN uses standard image Vision Transformers to learn video representations without 3D models or reconstruction. It treats videos as grids of frames and masks either spatial patches or temporal frames, then aligns the two views using a Siamese loss. This achieves top results on video benchmarks while needing 32-160x fewer training epochs than prior methods.

Ziyan Jiang, Jingbo Yang, Jiabao Ji et al.

Multimodal agents struggle to couple exploration and visual reasoning in 3D worlds—they can see anomalies or navigate, but struggle to do both together effectively, suggesting a fundamental gap in how these systems integrate action and perception.

WorldAuditBench is a benchmark for testing how AI agents find problems in 3D virtual worlds—like floating objects or walls you can walk through. It evaluates multimodal AI systems (vision-language models and vision-language-action models) on 213 anomaly detection tasks across 13 environments, measuring how well agents can explore systematically and visually identify issues.

Anson Y. Lam, Shuqing Li, Michael R. Lyu

Text-to-material generation can be more effective and controllable by outputting human-readable programs rather than raw images—users get both the final material and the explicit rules that construct it.

MatLoom generates realistic materials from text descriptions by creating compact, readable programs that define how layers combine to form appearance and physical properties.

Jaewoo Jung, Hyeonseo Yu, Honggyu An et al.

Training MLLMs to internally reconstruct 3D scene geometry (even in compact form) improves spatial reasoning and 3D understanding, suggesting that learning to imagine scenes is more effective than explicit geometric supervision alone.

This paper teaches multimodal AI models to reason about 3D scenes by first imagining a compact 3D representation before answering questions. Instead of relying on detailed geometric details, the model learns to assemble a coarse 3D layout from multiple viewpoints—similar to how humans understand 3D space—then uses this mental model to answer spatial reasoning questions more accurately.

Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini

Factorizing attention across different data dimensions improves crop segmentation, but dataset construction choices (tile size, class definitions) have outsized impact on results and must be standardized for meaningful model comparisons.

This paper introduces PAtteRNS, a transformer-convolutional model for crop segmentation in satellite imagery that separately applies self-attention to temporal, spectral, and spatial dimensions. The authors also highlight critical dataset issues—flawed class groupings and incompatible tile-size variants—that undermine fair model comparison and suggest standardization is needed.

Kuan-Po Huang, Haohe Liu, Puyuan Peng et al.

Emotion vectors in TTS models can be decomposed into a neutral-shift component and an emotion-specific component—controlling them separately via steering achieves much better emotion control than treating them as a single direction.

This paper improves emotional speech generation by decomposing emotion vectors into shared and residual components, then controlling them separately without retraining the model. The method, EmoRES, significantly outperforms prior vector steering approaches on multiple emotion metrics and human evaluation.

Hui Ren, Lei Fan, Henry Pao et al.

For long-video question answering, visually grounding object identity across time—not just retrieving relevant clips—is critical. GEB shows that organizing observations into entity biographies improves accuracy by 4+ points on day-long and week-long videos.

This paper solves a key challenge in long-video understanding: tracking the same physical object across hours or days of footage. The authors introduce Grounded Entity Biographies (GEB), which groups visual observations of the same object into retrievable "biographies" that preserve context.

Yijia Fan, Ziqi Huang, Zhongang Cai et al.

Unified models can learn to self-correct their outputs by applying RL to complete reflection loops, where both the reasoning about what's wrong and the actual image fixes improve together without needing external verifiers.

This paper presents UMM-Reflection, a method that teaches unified multimodal models to critique and fix their own image generations through reinforcement learning. Instead of just generating images once, the model can now look at what it created, identify problems, revise the image, and repeat—all within a single model.

Wenhao Li, Zhibin Wu, Chong Xiao et al.

Using LLM-generated semantics as a shared anchor point for aligning incomplete multimodal data is more robust than trying to reconstruct missing modalities or design complex fusion mechanisms.

SemMSA tackles multimodal sentiment analysis when some data is missing by using large language models to create rich semantic representations that ground all modalities together. Instead of reconstructing missing features, it aligns visual, acoustic, and text representations through spectral methods, achieving better results on standard benchmarks.

Debajyoti Mazumder, Mamta, Abhirama Subramanyam Penamakuri

Audio language models have a significant blind spot: they verify written facts reliably but fail on identical claims when spoken. Retrieval-augmented approaches help only when combined with explicit reasoning, not retrieval alone.

VeriSpeak is a benchmark for fact-checking spoken claims using audio language models. It contains nearly 4,000 spoken statements about real-world facts and tests whether models can verify claims directly from speech, especially when given retrieved text evidence.

Yuncong Yang, Jinlong Li, Yulong Xue et al.

Object-centric predictive models trained on multi-view video can enable underwater robots to plan manipulation tasks by imagining future states, without requiring expensive contact sensors or reconstruction.

This paper presents Underwater C³-JEPA, a world model that predicts how objects move during underwater robot salvage operations. Using multiple camera views and robot control signals, it learns to predict future object positions in a compact latent space without needing contact sensors. The model transfers well to downstream tasks and works on real underwater video.

Hong-Han Wang, Yuntao Wang, Hu Ding

Don't trust standard alignment scores as proof that multimodal models understand images—they can be fooled by the model's internal structure. Use task-based validation and the PA gap metric instead to verify genuine cross-modal integration.

This paper reveals that high visual-text similarity scores in multimodal AI models don't actually mean the model is properly integrating images and text. Using experiments across 13 models, the researchers show that replacing images with random noise barely changes similarity scores, even though task performance drops sharply.

Yuanhao Chen, Peter Chin

Audio language models don't reliably represent the same phonetic features in the same direction across speech and text—suggesting these models may process the two modalities quite differently despite using a shared decoder.

This paper investigates whether audio language models represent phonetic features consistently across speech and text inputs. Using minimal pairs of phonemes that differ in single features, researchers measured whether the same distinctive features (like voicing) are encoded in the same direction in both modalities across 6 models, 7 features, and 15 languages.

Yiran Wang, Xingyilang Yin, Junfu Pu et al.

This benchmark reveals that gameplay requires coordinating visual understanding, instruction following, and long-horizon planning—and shows clear performance gaps between model families, providing a standardized way to measure progress on embodied AI tasks.

GameHorizon is a comprehensive dataset and benchmark for evaluating AI models on video game tasks. It includes 5,000 hours of gameplay from 21 AAA games with aligned videos, actions, and instructions at multiple time scales, plus offline and online evaluation tracks to measure how well models understand and execute gameplay across different planning horizons.

Wangbo Yu, Kunhao Liu, Wenbo Hu et al.

By storing observations in a viewpoint-aware implicit 3D memory rather than explicit depth maps, video world models can generate longer, more consistent videos across different camera angles without running out of token budget.

WorldCrafter is a video world model that maintains consistent 3D-aware memory across different camera viewpoints and long time horizons. It uses an implicit memory system that compresses multi-view observations into tokens optimized for the requested viewpoint, enabling realistic minute-long video generation from a single image or text prompt while respecting what was seen before.

Haoran Yuan, Zekai Wang, Boning Shao et al.

Tactile sensing matters for dexterous manipulation: modeling contact evolution as part of world state (not just as input) nearly triples performance, and pretrained vision models can efficiently extend to touch with minimal data.

DexTacWAM combines vision and touch sensing for robot hand manipulation by extending video prediction models with tactile data from fingertips. The system predicts both visual and contact dynamics together, achieving 70% success on complex multi-finger tasks—nearly double the best existing approach—while learning from minimal touch data by adapting a pretrained vision model.

Muzhe Wu, Zuchen Li, Xu Wang et al.

Contextualizing visual instructions to match a user's real workspace—through AI-generated images and videos—improves task performance and trust compared to generic pre-authored tutorials.

This paper presents a system that generates live visual instructions tailored to a user's specific workspace and task progress, rather than showing pre-recorded tutorials. Using AR and generative AI, it creates goal images and demo videos that match the user's actual environment, helping them complete physical tasks more accurately and confidently.

Taoyong Cui, Zhongyao Wang, Xinyue Xu et al.

A single learning principle based on factorized predictions can work across radically different domains, suggesting world modeling doesn't need domain-specific architectures—and can even guide real scientific discovery.

JEPA-Anything is a unified framework for building predictive models across completely different domains—from videos to molecules to weather—using a technique called orthogonal predictive factorization. Instead of training separate models for each domain, it learns to decompose predictions into independent factors that work across vision, biology, physics, and clinical data.

Tica Lin, Deepak Chandran, Gauri Jagatap et al.

A shared, human-readable schema can simultaneously ground agent generation and enable human interpretation, making AI-generated content more transparent and controllable.

This paper introduces a semantic action graph—a structured representation of sports matches using performers, actions, recipients, moments, and states—that enables both AI agents to generate video highlights and humans to understand and control them.

Hanchu Zhou, Brendan Lynch, Raman Goyal et al.

By treating vision and tactile signals at different timescales in a lightweight architecture, you can build fast tactile-aware robot controllers (11.9ms latency) that outperform larger models on real contact-rich tasks.

This paper presents Agile-WAM, a tactile world action model that predicts future robot states and actions for contact-rich manipulation tasks.

Haocheng Xi, Yiming Xie, Hexu Zhao et al.

Hybrid attention architectures that combine local Softmax with linear memory can dramatically accelerate video generation without sacrificing quality—enabling practical real-time video synthesis on modern hardware.

Video DeltaNet combines local attention with efficient linear memory to speed up video diffusion models. By mixing Softmax attention for fine details with a new Video Delta Attention mechanism for long-range context, it generates high-quality videos 14.5x faster than baseline models while maintaining visual quality.

Hassan Saeed Hassan Albattra, Mazen Mohammed Bahgat, Rahatara Ferdousi et al.

Multilingual healthcare AI needs explicit testing for how language, communication style, and missing information affect risk assessment—aggregate accuracy scores alone don't catch safety failures that emerge in specific languages or contexts.

HerHealthEval is an evaluation framework that tests how well AI language models understand women's health concerns across multiple languages (English, French, Arabic) and communication styles (clinical, casual, emotional, vague).

Sara Pieri, Evangelos Kazakos, Shizhe Chen et al.

Combining dense captioning with pixel-level grounding requires jointly optimizing text generation and mask selection—PANORAMA shows this can be done effectively by conditioning a segmenter on phrase representations and learning which masks correspond to each phrase.

This paper introduces PANORAMA, a vision-language model that generates detailed image captions while simultaneously grounding each phrase with pixel-level segmentation masks.

Guanhua Ji, Tianyu Li, Dayoon Suh et al.

Combining generated video with generated audio allows robots to infer not just motion but also the forces needed for contact-heavy tasks—something video alone cannot provide.

This paper shows how robots can learn contact-rich manipulation tasks by generating both video and audio together. The system uses the loudness of generated contact sounds to create force profiles that guide the robot's movements, enabling tasks like pushing and grasping that require precise force control. The approach works without task-specific training data.

Chuhao Chen, Peter Wonka, Chaoyang Wang et al.

This model enables mid-generation control of video generation using physics-grounded motion signals rather than pixel positions, making it possible to interactively steer object dynamics while the video is being generated.

PhysStream generates videos from images while letting you control object motion in real-time using physics-based signals. Instead of specifying exact positions, you provide velocity changes that the model interprets as physical dynamics. It uses scene memory (tracking where objects are) to maintain consistency across frames, enabling interactive control over multi-object scenes during generation.

Jahanvi Rajput, Dhruv Kudale, Saikiran Kasturi et al.

Processing tables as structured text instead of images makes them easier for LLMs to understand, scales better across languages, and achieves better performance on table QA tasks without needing visual encoders.

This paper presents DELTA and TARQA, a two-stage approach for table understanding that converts table images into structured text (OTSL format) rather than relying on vision-language models. DELTA handles table structure recognition and OCR, while TARQA is an LLM fine-tuned to answer questions about tables in this text format.

Yunfei Ge, Anbang Liu, Qineng Wang et al.

Foundation models excel at metric spatial reasoning (distance, angles) but fail at topological reasoning—a more fundamental aspect of spatial understanding. This gap persists even with fine-tuning and reinforcement learning, suggesting current architectures lack core spatial intuition.

MindTopo is a benchmark testing whether AI models can reason about topological properties—spatial relationships that stay the same under stretching or bending. It evaluates 14 multimodal AI models on five topological concepts (continuity, separation, order, enclosure, knots) through reasoning tasks and embodied planning.

Yingzhi Wang, Reem Alhazzani, Muhammad Alqurishi

Arabic speech AI was severely underrepresented in multilingual models—this project provides the dataset, trained models, and evaluation tools needed to build general-purpose Arabic speech systems from scratch.

Nuha-Speech addresses the lack of Arabic speech AI models by creating a comprehensive framework: a 1.5M-sample Arabic speech question-answering dataset, fine-tuned models based on Qwen-Omni, and evaluation benchmarks. This work establishes foundational infrastructure for building Arabic-capable speech AI systems despite limited available resources.

Bowen Zhang, Hsiu-Wen Cheng, Hongyu Yang et al.

Foundation models for time-series forecasting need task-specific fine-tuning to work well for glucose prediction, and multimodal dietary data (food images + nutrition) provides clinically useful signals that significantly improve postprandial glucose forecasting.

This study evaluates whether time-series foundation models can forecast blood glucose levels from continuous glucose monitoring (CGM) data, and whether adding dietary information improves predictions.

Lisa Bylinina

Visual grounding helps language models learn concrete object properties, but this benefit doesn't show up in standard benchmarks—suggesting we need better evaluation methods to measure what models actually learn from multimodal information.

This paper tests whether giving a small language model visual grounding for words (like showing it images of 'banana') before training helps it learn language better. The author finds that visual initialization leaves a lasting imprint on the model, especially for object-property knowledge, but this advantage is invisible to most standard benchmarks that test grammar and abstract reasoning.

Anqi Li, Yuxin Chen, Zhaobo Li et al.

End-to-end vision-language models can control complex humanoid robots for real-world navigation by learning whole-body coordination in simulation, eliminating the need for modular planning pipelines and real-world data collection.

TANGO is a vision-language model that enables humanoid robots to navigate cluttered indoor spaces by predicting full-body joint movements directly from natural language instructions and camera images. Unlike traditional 2D path planning, it coordinates arm placement, torso adjustment, and walking patterns to move through complex 3D environments.

Siting Li, Zhengyang Wang, Simon Shaolei Du et al.

When building multimodal models, image tokenizer choice matters not just for image quality but for how well it helps the model learn text and vision together—and the best tokenizer for one task may not be best for another.

This paper studies how image tokenizers work in multimodal AI models by building a controlled training setup that tracks how well different tokenizers perform across text, image generation, and image understanding tasks.

Tobias Susetzky, Raphael Rehms, Dmitrii Seletkov et al.

This is the first generative model to handle the full complexity of real patient data (multiple modalities, irregular timing, uncertainty), enabling both predictive tasks and counterfactual reasoning for personalized medicine.

NOAH is a generative AI model that learns from complete patient medical histories to predict future health outcomes. It processes diverse data types—images, vital signs, lab results, and clinical notes—across irregular time intervals, enabling doctors to forecast patient trajectories, simulate treatment scenarios, and identify disease risks.

Xiaofu Chen, Stella Frank, Yova Kementchedjhieva

Vision encoders learn and store conceptual semantic information like canonical colors independently of visual input, and this knowledge is tightly linked to object recognition—providing a measurable way to understand what abstract concepts models actually learn.

This paper investigates whether vision encoders in vision-language models (VLMs) learn conceptual information beyond what's visible in images. Using canonical color (like 'bananas are yellow') as a test case, researchers found that models can decode object colors from grayscale images, suggesting they learn abstract semantic knowledge.

Yankai Fu, Ning Chen, Junkai Zhao et al.

Adaptive tactile fusion and joint visual-tactile prediction enable VLA models to handle contact-rich manipulation where vision alone fails, achieving 71% success on dexterous tasks.

DeCAL is a vision-language-action model for robot hands that combines visual and tactile (touch) sensing to perform complex manipulation tasks. It uses specialized AI experts working together to understand scenes, imagine future states, and generate actions—all while explicitly modeling contact dynamics that cause visual occlusions during dexterous manipulation.

GeonU Kim, Shin Dong-Yeon, Tae-Hyun Oh

You can generate realistic novel views of mirror scenes by treating reflections as virtual views and using gated attention mechanisms—no retraining needed, just clever use of existing diffusion models.

This paper presents Ref-GeNVS, a method for generating novel views of scenes containing mirrors without requiring additional training. The key innovation is treating mirror reflections as complementary views by estimating the mirror plane and reflecting camera poses, then using a two-stage approach with special attention mechanisms to ensure reflections stay consistent during generation.

Vivek Chavan, Pengtao Xie, Yahuan Shi et al.

Visual distractors cause robot policies to select wrong objects not because they lose manipulation skills, but because they struggle to ground attention on the correct target—a problem that can be fixed by explicitly training better target selection.

This paper identifies why robot learning policies fail when similar-looking objects are present, even though they work well normally.

Ye-Chan Kim, Seunghee Choi, SeungJu Cha et al.

Using a VLM to detect actual visual transitions in videos produces better event localization and captions than pre-written transition descriptions inserted at fixed positions.

This paper tackles dense video captioning—describing multiple events in long videos—by using a vision-language model to intelligently detect transition moments between events rather than blindly inserting captions everywhere.

Sixu Yan, Shikang Wang, Binhua Huang et al.

Decoupling physical grasp synthesis from task-dependent reasoning allows robotic systems to leverage foundation model improvements without retraining, and enables cross-hand generalization through explicit kinematic and stability constraints.

AdaRoboVLG combines vision-language models with robotic grasping by separating physical grasp synthesis from task understanding. A base policy generates and evaluates grasp candidates using kinematics and stability checks, while foundation models provide task-specific context.

Tingyu Song, Mingxin Li, Yanzhao Zhang et al.

Distilling ranking knowledge from a reranker into embedding models significantly improves compositional reasoning—the ability to distinguish scenes by attribute-object relationships—achieving 82.7% on compositional benchmarks while maintaining standard retrieval performance.

This paper improves how multimodal AI models understand complex visual scenes by teaching embedding models to better distinguish between scenes with the same objects but different arrangements.

Vasileios Baltatzis, Mert Inan, Connor Gillis et al.

Sign language translation needs discourse context to maintain spatial consistency and entity tracking across multiple sentences—not just translating each sentence independently.

DiscoSign translates English text to American Sign Language glosses while tracking discourse-level phenomena like spatial references and entity consistency. Traditional sign language systems work sentence-by-sentence, missing how entities and concepts maintain meaning across longer conversations.

Haoyuan Deng, Haichao Liu, Wenkai Guo et al.

By explicitly modeling contact forces alongside visual and kinematic information, this system achieves 82% success on precision assembly tasks—a massive jump from 15% for existing approaches—showing that understanding physical interaction is critical for real-world robot manipulation.

Facet-0 is a robotic foundation model that learns to perform precise assembly tasks by predicting how its actions will affect contact forces with objects. It combines vision, language, and force feedback to generate robot movements that maintain sub-millimeter accuracy during assembly, trained on 1,000 hours of real robot data across multiple manufacturing setups.

Shiao Xie, Siyu Chen, Jianwei Lv et al.

Medical AI needs dual optimization: factual correctness (verifiable through evidence) and patient communication quality (context-dependent). G-CARL shows that structured checklists paired with retrieval-based verification can train models for both simultaneously better than standard approaches.

This paper introduces a new task where AI systems explain medical reports to patients in accurate, accessible language. The key innovation is G-CARL, a training method that uses retrieval-based fact-checking and customized checklists to ensure explanations are both medically accurate and responsive to what patients actually want to know, without limiting creative variation in responses.

Anton Lambrecht, Reda El Hail, Xianjun Jiao et al.

Different RF sensing technologies excel at different things: IR-UWB wins on accuracy, FMCW on generalization to new spaces. For healthcare monitoring, choose based on whether you prioritize activity recognition or robustness to environmental changes.

This paper compares three radar technologies (FMCW, IR-UWB, and Wi-Fi) for detecting human activities and sleep patterns from ceiling-mounted sensors. Using the same neural network and test conditions across 20 people and different room layouts, the study shows IR-UWB performs best overall (89% accuracy), while FMCW adapts better to new environments.

Narges Ahmadi, Yubo Jiao, Jônatas Augusto Manzolli et al.

Large language models can match or exceed traditional machine learning for travel behavior prediction without task-specific training, and adding visual context from survey images improves performance—showing that multimodal AI can enhance behavioral modeling when integrated with human-centered ...

This paper presents a three-agent workflow that combines chatbot surveys, data processing, and prediction to model how weather affects commuter mode choices. The system collected 454 survey responses about travel preferences across different weather scenarios, then compared traditional statistical models with nine different large language models (2-35B parameters) for predicting travel behavior.

Yu Chen, Ting Lei, Yaoyi Li et al.

Breaking down rule-following tasks into separate perception, rule interpretation, and planning steps helps multimodal models generalize better to new rules and constraints than end-to-end approaches.

This paper introduces RuleMaze, a benchmark for testing whether multimodal AI models can navigate mazes while following natural-language rules. The authors propose a method called Disentangled Multimodal Planning that breaks down the task into separate steps—understanding the scene, interpreting rules, and planning actions—to help models follow complex constraints more reliably.

Mosharof Hossain, Md Rabiul Islam, Limon Halder et al.

Prompts can be integrated deep into image segmentation networks through channel-wise attention, rather than just at the end, making models better at finding anatomical structures across different medical imaging modalities.

This paper introduces PCCA, a mechanism that uses text prompts to guide how neural networks process medical images at multiple levels, improving segmentation accuracy across different body parts and imaging types. The method adaptively adjusts which image features matter most based on the prompt, achieving 10-23% improvements over standard approaches.

Ingo Marquardt, Anthilia Alchanat, Priyanka Jain

Silent reading produces decodable word-level information in EEG that improves with more training data—this opens a scalable path to studying how the brain processes language without relying on unreliable self-reports of inner speech.

Researchers decoded which words people were silently reading from brain activity (EEG), using 49 hours of data from one participant. They trained a neural network to match EEG signals with word embeddings from a language model, achieving above-chance accuracy on 240,000 word presentations.

Haoxiang Cao, Jiajiong Cao, Xuanpu Zhang et al.

By decomposing credit assignment across both pipeline stages and within the planner's reasoning trace, DARS enables more efficient training of complex multi-stage image editing systems compared to naive joint optimization.

DARS is a reinforcement learning system for training instruction-based image editing that uses a two-stage pipeline (planner + renderer). It solves the credit assignment problem—figuring out whether failures come from bad planning or bad rendering—using structured reasoning outputs and multi-level reward analysis to provide targeted feedback to each component.

Zhenyao Cui, Siyuan Kan, Siyang Li et al.

Brain-based image retrieval can work for new users without retraining by learning to recover the geometric transformation between their brain's coordinate system and a shared visual space, using only unlabeled test-time alignment.

This paper tackles cross-subject EEG-to-image retrieval—retrieving images that match brain signals from new users without labeled training data. The key insight is that different people's brains organize visual concepts similarly but along different coordinate directions.

Zhenyao Cui, Siyuan Kan, Dingkun Liu et al.

Brain-to-image decoding can work with far fewer repetitions by anchoring both neural and visual representations to a shared reference point, rather than just denoising the query alone.

This paper tackles brain-to-image retrieval with limited neural data. Current methods require averaging 80+ brain scans per image, but the authors show that low-repetition queries fail not just due to noise, but because brain and image representations misalign.

Lei Bai, Jiaqi Cao, Chiyu Chen et al.

This work demonstrates how to build AI agents for science by combining multimodal pre-training with agentic reinforcement learning and memory-augmented architectures, achieving strong performance on scientific reasoning and forecasting tasks without requiring task-specific model modifications.

Intern-S2-Preview is a large multimodal AI system designed to tackle scientific discovery tasks by reasoning over diverse data types, using scientific tools, and working across long-horizon problems.

Yi-Chung Chen, Philip Jacobson, Tom Lampo et al.

Using physical trajectory data as training supervision helps video embedding models better understand motion-centric driving events, improving retrieval accuracy by 5-10% while keeping inference simple and efficient.

This paper tackles retrieving relevant driving video clips from large datasets by improving multimodal embedding models. The key innovation is TraVEL, which fine-tunes video embeddings using trajectory (vehicle motion) as training supervision, helping the model understand motion-centric events like turning or accelerating.

Daniel Perkins, John Squires, Janou Milligan et al.

Instead of training separate models for each domain, you can use an MLLM router to dynamically select which vision backbone handles each image, getting both better generalization and easier updates without retraining.

ARMDIL uses a multimodal language model to intelligently route images to the best-suited vision model (CNNs, self-supervised learners, or vision-language models) within an ensemble.

Serli Kopar, Sam Gijsen, Abner Hernandez et al.

Speech-based disease detection models may not be learning genuine disease characteristics but rather dataset-specific patterns, raising serious concerns about their reliability for real-world clinical use.

This paper investigates whether speech models trained to detect Parkinson's disease actually learn disease-specific patterns or just exploit dataset quirks.

Muhammad Hannan Akram, Muhammad Abubakar Rashid, Wassi Haider Kabir et al.

Heterogeneous AI agents can efficiently synchronize their understanding in distributed networks by translating belief updates through edge-deployed models, without requiring shared training or identical architectures.

This paper proposes a framework for AI agents in 6G networks to synchronize their understanding (beliefs) despite using different AI models and operating under different constraints.

Chuyue Li, Jinpeng Yu, Haozhe Wang et al.

Agents can now learn from and generate high-quality videos by working with structured knowledge graph representations instead of raw pixels, improving video generation quality by 20.7% over existing methods.

AVA-Encoder learns video representations as knowledge graphs that agents can reason about and edit. It converts videos into structured text and asset layers, then reconstructs videos from these representations. A natural-language feedback loop optimizes the encoding, enabling agents to work with cinematic-quality videos more effectively.

Yan Deng, Fei Xu

For embodied AI agents navigating from visual instructions, explicitly modeling temporal context (past-only memory), multi-step planning with single-step execution, and decoupled termination detection significantly improves navigation success and efficiency.

DreamFly improves aerial drone navigation by combining three key techniques: a causal memory system that uses only past observations to avoid information leakage, a receding-horizon planning approach that predicts multiple future actions but executes one at a time, and explicit stop detection from action predictions.

Pedro Sousa, Will Tebbutt, Sadiq Jaffer et al.

Foundation models trained on satellite imagery can capture persistent surface properties (terrain, vegetation, water) that explain local weather variations better than hand-crafted features, enabling more accurate probabilistic weather predictions at arbitrary locations.

This paper shows that Earth observation embeddings from satellite data can improve weather downscaling—predicting local weather at specific locations from coarse global weather models.

Weihao Bo, Shan Zhang, Yanpeng Sun et al.

Current multimodal AI models excel at understanding diagrams visually but struggle significantly with converting them to executable code, suggesting this is a critical gap to address for scientific writing tools.

Diagram-MMU is a benchmark with 3.7k scientific diagrams and 18.3k questions that tests how well AI models can understand diagrams and convert them to code. The benchmark evaluates 12 models on three tasks: turning diagrams into LaTeX code, editing diagram code, and answering questions about diagrams—revealing that code generation is much harder for models than visual understanding.

Changhao Xiang, Shangyu Xing, Zhen Wu et al.

By interleaving visual objects directly into text during pretraining, you can teach multimodal models object-level grounding 12x more efficiently than traditional image-text pair training.

This paper introduces MultiModal Code-Switching (MMCS), a new way to train vision-language models by replacing words in text with their corresponding visual objects. Instead of just pairing whole images with descriptions, MMCS explicitly shows the model which objects match which words, making training much more efficient—achieving the same performance with 12x less data.

Oluwanifemi Bamgbose, Simon Rosen, Jash Shah et al.

Automated TTS evaluators need to move beyond generic 'naturalness' scores and evaluate specific linguistic dimensions of speech quality, as current tools miss many errors that human listeners easily detect.

This paper reveals that current automated Text-to-Speech evaluation tools (MOS predictors and Audio-LLM judges) fail to capture the full range of speech quality issues that humans perceive.

Hunar Batra, Lachin Naghashyar, Ashkan Khakzar et al.

Sparse autoencoders can turn multimodal model internals into controllable feature interfaces: you can identify what changed during multimodal training, find features causing specific behaviors, and steer or remove them to improve safety or task performance.

This paper introduces MMDiff, a framework that uses sparse autoencoders to decompose multimodal language models into interpretable features.

Diandian Zhang, Tingyu Song, Lin Fu et al.

Visual realism in video generation doesn't guarantee scientific correctness: models that look good often fail at capturing accurate scientific and causal dynamics, a gap that needs targeted evaluation and improvement.

Sci-VBench is a benchmark with 1,253 expert-annotated examples for evaluating how well AI models generate videos that require scientific knowledge and reasoning across 60 subjects in science, healthcare, humanities, and engineering.

Sarvesh Baskar, Zikui Cai, Shayan Shabihi et al.

Video models fail not because they can't see events, but because they can't reliably track and count them over time—adding more frames helps slightly but doesn't fix the core temporal reasoning problem.

Video language models struggle with counting events in videos, especially when events happen frequently or repeatedly.

Zhiheng Wang, Bo Peng, Lai Wei et al.

Visual tool-use in multimodal models often creates an illusion of improvement: while aggregate accuracy may increase, the returned visual evidence frequently has no causal effect on the final answer, making these expensive operations ineffective for most queries.

This paper investigates whether multimodal AI models actually benefit from visual tool-use operations like cropping and zooming. Using causal analysis, the authors discover that these operations often don't meaningfully improve answers despite higher computational cost—models either ignore the visual evidence they retrieve or use it incoherently.

Yuanjing Xu, Xinyan Liu, Weidong Chen et al.

Adding depth information significantly improves object counting in crowded scenes—the method reduces counting errors by 62% compared to RGB-only approaches, showing that 3D spatial cues are crucial for handling occlusions.

This paper tackles counting objects in crowded, occluded video scenes by combining RGB images with depth information. The method uses a depth-guided detector with cross-attention between color and depth data, plus occlusion prediction, to better identify individual objects even when they overlap. The authors also release a new RGB-D dataset for this task.

Christo Kurisummoottil Thomas, Omar Hashash, Walid Saad

Networks can become active intelligence coordinators for physical AI by deploying reasoning agents (digital twins) that share spatiotemporal context through causal reasoning and transmit only beliefs with cognitive value, rather than optimizing for throughput alone.

This paper proposes holonic digital twins (HDT-Nets)—intelligent network agents that actively reason about their environment rather than passively mirroring physical systems.

Ayoub Kirouane, Christos Petrocheilos

Domain-specific fine-tuning dramatically improves retrieval and generation for underrepresented languages: a modest 1B embedder outperforms multilingual models when trained on just 65K in-domain Greek examples, showing that language adaptation is more important than model size for specialized tasks.

This paper adapts NVIDIA's Nemotron retrieval system for Modern Greek across legal, energy, financial, and medical domains. The authors mine Greek corpora, train specialized retrieval models, and create HERA—the first large-scale Greek RAG benchmark.

Aniri, Jinhe Bi, Peng Liao et al.

Modality imbalance (text drowning out vision) is a real bottleneck in multimodal reasoning. By detecting when visual input is being ignored and training only on those critical tokens, you can make self-distillation much more effective.

This paper identifies and addresses modality imbalance in multimodal language models—where text dominates over visual information during reasoning. OPD-V uses positive and negative teacher models with modified images to detect when the model isn't properly using visual input, then selectively applies self-distillation only on tokens where visual information matters most.

Yuezhang Peng, Yuxin Liu, Changfeng Gao et al.

Framing spoken language understanding as structured function calling—rather than slot-filling—lets audio language models generalize to new tasks without retraining, similar to how code models handle function calls.

This paper introduces Spoken Function Calling (SFC), a new way to understand spoken language that treats semantic extraction like function calls with structured definitions.

Yang Yang, Qinyu Zhao, Mouxiang Chen et al.

By sharing backbone parameters across parallel branches with task-specific computation allocation, you can improve multimodal model performance without increasing model size or inference latency.

ParVL introduces a framework for scaling multimodal AI models by running multiple parallel vision and language processing branches that share the same core parameters. Instead of making models bigger or slower, it reuses existing components more efficiently and lets different tasks use different amounts of vision vs.

Zhen Fang, Yu Zeng, Wenxuan Huang et al.

Multimodal agents need explicit architectural constraints to use visual tools and external knowledge rather than defaulting to text search and internal memory; decoupled perception-exploration pipelines with staged tool unlocking significantly improve performance.

Video-DeepResearch extends multimodal AI agents to handle continuous video streams with web search integration. The system addresses two key problems: agents ignoring visual information in favor of text search, and relying on memorized knowledge instead of actually using tools.