Recent AI research papers with accessible summaries. Updated daily from arXiv, summarized for developers who don't read papers regularly.
Yuxin Jiang, Wenqiang Sun, Jiaqi Li et al.
Pretrained video generators can serve as powerful priors for discovering latent actions from visual transitions, enabling few-shot and zero-shot action transfer across different visual contexts and robot embodiments.
This paper proposes O²LaF, a method for learning actions from unlabeled videos by repurposing pretrained video diffusion transformers. The approach uses a single pretrained model to initialize both inverse dynamics (what action caused this change?) and forward dynamics (what happens after this action?), learning transferable actions without manual labels.
Jiahao Huang, Jun Wang
Vision-language models exhibit selective concept access similar to human consciousness theories—information becomes available to different parts of the model based on what's needed, and this selective access develops as a learned capability during training.
This paper introduces MC-Lens, a tool for understanding how vision-language models process and access concepts across visual and textual information. Using global workspace theory as inspiration, the researchers show that VLMs selectively access concept information based on task needs, and that this information flows through downstream computations.
Dayoon Ko, Yichuan Wang, Soyeong Jeong et al.
By reusing computations across incrementally-changing screenshots, you can index screen history 3.7x faster in real-time without sacrificing visual understanding—making local visual assistants practical.
StreamEmbed enables real-time visual indexing of screenshots by exploiting pixel redundancy across frames. Instead of recomputing embeddings from scratch, it uses spatial token pruning to skip unchanged image blocks and temporal key-value reuse to cache computations across screenshots, achieving 3.7x faster encoding than standard visual encoders while maintaining accuracy.
Ellis Brown, Chris Hoang, Shengyi Qian et al.
For building multimodal models, dedicated vision capacity trained jointly with the language model is more important than using pretrained vision encoders, and this capacity can be efficiently placed early in the backbone rather than as a separate pre-fusion module.
This paper investigates how vision encoders should be designed in multimodal language models by systematically comparing three design choices: using a pretrained visual encoder, dedicating capacity to vision processing, and where to place that capacity.
Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov et al.
A single small embedding model can serve multiple low-resource languages effectively through targeted fine-tuning with synthetic data, but scaling up to larger multilingual models remains necessary for competitive performance on these languages.
Researchers fine-tuned a small 300M embedding model (EmbeddingGemma) to work across four Central Asian languages—Tajik, Kazakh, Uzbek, and Kyrgyz—using synthetic question-answer pairs generated by a language model. The fine-tuned model significantly improved retrieval performance (14-26 percentage points) on a multilingual benchmark, though it still lags behind larger 560M multilingual models.
Weizhong Huang, Huaijia Kou, Tianyu Xie et al.
By aligning KV cache compression with semantic planning constraints, you can reduce memory usage by 3.35x and speed up video generation by 1.22x without retraining, making agentic video generation more practical for resource-constrained settings.
Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee et al.
Reasoning in continuous latent space instead of discrete tokens unlocks test-time scaling and better controllability for image generation in multimodal models, showing 7% improvement on GenEval benchmarks.
This paper introduces LARC, a framework that enables unified multimodal models to perform reasoning directly in continuous space rather than discrete tokens, bridging the gap between language-based reasoning and image generation.
Jiongfan Yang, Weichen Fan, Haiwen Diao et al.
By splitting image frequency components across local and global attention paths, you can generate finer image details without the quadratic cost of full global attention—achieving better quality at lower computational cost.
SpecSlot is a pixel-space image generation model that separates low-frequency and high-frequency image details into different processing paths. Low-frequency information uses global attention for context, while high-frequency details are processed locally, reducing computational cost while maintaining image quality.
Yuhan Liu, Lianhui Qin, Shenji Wan
By combining diffusion-based visual layout generation with geometric constraint-solving, SceneBloom creates training environments so realistic that policies trained on them outperform those trained on real scans—solving a key bottleneck in embodied AI.
SceneBloom is an AI system that generates realistic 3D indoor environments for training embodied agents (robots/AI that interact with physical spaces). It combines visual layout generation with geometric reasoning to create physically valid room layouts with furniture arrangements, achieving better realism and stability than existing methods.
Dianye Huang, Jiamei Guo, Yue Zhou et al.
Combining diffusion-based continuous prediction with language models outperforms discrete token prediction for medical measurements, suggesting hybrid architectures better match the continuous nature of real-world quantitative tasks.
This paper presents HAD-VLM, a medical AI system that combines language understanding with precise numerical measurement from ultrasound and X-ray images.
Ailiang Lin, Zhuoyun Li, Yusong Wang et al.
Language-specific noise in LLM embeddings can be removed at inference time using paired multilingual sentences, significantly improving cross-lingual and monolingual embedding quality without any training.
This paper tackles multilingual text embeddings from frozen language models by identifying that semantically equivalent sentences misalign across languages due to language-specific geometry.
Bowei Liu, Xinchen Zhang, Xuhuan Li et al.
Graded, component-level verification with pairwise comparison significantly outperforms binary verification for evaluating complex multimodal outputs, especially in structured reasoning tasks where compositional correctness matters.
This paper introduces OmniVerifier-C1, a visual verifier that moves beyond simple True/False judgments to provide detailed, component-level scores for evaluating generated images.
Bhanu Tokas, Rahul Nair, Rockwell Jackson et al.
You can now explain any vision model's decisions using concepts without needing model internals or expensive labeled data—just show the model what happens when you hide concept regions.
Hide and Seek is a method for explaining black-box vision models using human-understandable concepts. Unlike existing approaches, it doesn't require access to model weights or labeled concept datasets. The method finds concept-relevant images, masks the concept regions, and measures how the model's output changes to determine what concepts influence predictions.
Joseph Lee, Yanjing Li, Bo Qiang et al.
Natural language is now a practical interface for protein design—you can describe desired protein properties in text and get back 3D structures, with experimental validation showing it works for real-world targets.
Prompteina is an AI model that generates protein structures from natural language descriptions, similar to how text-to-image models work. Trained on 1.8M protein structures with detailed captions, it can design proteins with specific structural and functional properties without needing separate input formats for each goal.
Qiushi Han, Keya Hu, Linlu Qiu et al.
Adding a simple visual memory system that preserves and lets models retrieve past observations dramatically improves multimodal models' reasoning in interactive visual environments—achieving perfect performance on challenging puzzle games.
VISTA is a visual harness that enhances multimodal models' ability to solve complex interactive visual tasks by giving them long-horizon vision and lossless visual memory. The system lets models directly perceive environments, store past observations, and actively retrieve them while reasoning.
Jiahan Zhang, Chaohao Yang, Namitha Guruprasad et al.
By lifting 2D video controls into explicit 3D space, you can resolve ambiguities in object motion and create videos where camera and object movements are geometrically consistent—a major improvement over 2D trajectory-based video control.
GenCine enables artists to control video generation by editing 3D camera paths and object motions in a scene scaffold, rather than using ambiguous 2D trajectories. The system projects these 3D controls into guidance maps that a pretrained video model learns to follow, producing videos with consistent camera-relative motion and improved geometric coherence.
Kunxiong Zhu, Zhihao Shu, Hangyu Zheng et al.
Multimodal LLM serving requires rethinking GPU resource allocation around the Encode stage—treating it as a bottleneck control point rather than a separate service unlocks significant throughput gains.
EAServe optimizes serving multimodal LLMs by treating the Encode stage as a control point for the three-stage Encode-Prefill-Decode pipeline. It uses adaptive micro-batching, partial offloading, and GPU partitioning to balance resource utilization across stages, achieving 4.3x higher throughput than existing systems under latency constraints.
Wenhao Li, Zhibin Wu, Chong Xiao et al.
Using LLM-generated semantics as a shared anchor point for aligning incomplete multimodal data is more robust than trying to reconstruct missing modalities or design complex fusion mechanisms.
SemMSA tackles multimodal sentiment analysis when some data is missing by using large language models to create rich semantic representations that ground all modalities together. Instead of reconstructing missing features, it aligns visual, acoustic, and text representations through spectral methods, achieving better results on standard benchmarks.
Ji Xie, Dewei Zhou, Xinyu Huang et al.
By combining real image supervision with pure-color anchors and a shared hex-prompt interface, you can now control exact object colors in both image generation and editing tasks with professional-grade precision.
Paint-Anything enables precise color control in AI image generation and editing by letting users specify exact colors using hex codes (like #FF5733). The system learns to understand hex values through a training dataset of 500K images with color labels and pure-color reference images, achieving much better color accuracy than existing methods.
Taoyong Cui, Zhongyao Wang, Xinyue Xu et al.
A single learning principle based on factorized predictions can work across radically different domains, suggesting world modeling doesn't need domain-specific architectures—and can even guide real scientific discovery.
JEPA-Anything is a unified framework for building predictive models across completely different domains—from videos to molecules to weather—using a technique called orthogonal predictive factorization. Instead of training separate models for each domain, it learns to decompose predictions into independent factors that work across vision, biology, physics, and clinical data.
This paper presents SemanticKV, a method to compress the key-value cache in video generation models by using semantic constraints from a planner to decide which cached information to keep.