Papers

Uwe M. Borghoff, Paolo Bottoni, Remo Pareschi

Hardware-enforced coordination can provide hard safety guarantees for autonomous systems by implementing coordination rules at the FPGA level, separating deterministic interaction management from adaptive AI reasoning.

This paper proposes using FPGAs to enforce safety-critical coordination rules directly in hardware for autonomous systems that combine AI models with real-time control. By moving coordination logic from software to hardware, the system guarantees deterministic timing and verifiable safety constraints while keeping AI reasoning flexible.

Giovanni Monea, Nathan Godey, Kianté Brantley et al.

Splitting Transformer computation into separate streams for token prediction and state maintenance improves both training efficiency and model performance—a simple architectural change with consistent gains across scales.

This paper proposes separating two functions in Transformers: predicting the next token and maintaining state for future predictions. The authors design a dual-stream architecture and show it improves language modeling efficiency and downstream task performance by 2-3% compared to standard Transformers.

Patrick Podest, Marco Pichler, Elias Bürger et al.

TiRex-2 enables efficient streaming multivariate forecasting with constant per-patch inference cost and zero-shot generalization, solving the quadratic complexity problem of Transformer-based time series models.

TiRex-2 is a time series foundation model built on xLSTM that handles multiple variables and streaming data efficiently. Unlike Transformer-based models that slow down with longer sequences, TiRex-2 uses a memory-based recurrent design that processes new data at constant cost, even when variables evolve together and some future values are known in advance.

Jian Gu, Aldeida Aleti, Chunyang Chen et al.

SemRF provides a principled way to track semantic meaning across model layers, enabling researchers to distinguish real computation from measurement drift and connect layer-wise complexity to parameter efficiency.

This paper introduces Semantic Reference Frames (SemRF), a mathematical framework for analyzing how language models process information across layers.

Shun Lei, Huaicheng Zhang, Dapeng Wu et al.

For music generation at scale, separating semantic planning (what to generate) from acoustic refinement (how to generate it) and training them sequentially rather than simultaneously improves both coherence and sound quality.

LeVo 2 generates full-length songs by combining language models and diffusion models in a hierarchical approach: first predicting mixed vocal-instrument tokens for overall coherence, then refining each track separately for acoustic detail.

Liyao Wang, Ruipu Wu, Haojun Xu et al.

Combining explicit 3D geometry (camera poses, spatial relationships) with visual matching dramatically improves cross-view localization and enables zero-shot transfer between ground and drone views without paired training data.

This paper tackles cross-view object geo-localization—finding a target object in satellite imagery when given a ground or drone photo. The authors introduce a large dataset with 220K+ image pairs and geometric metadata, plus GAGeo, a unified framework that predicts object locations, masks, and camera poses simultaneously using 3D spatial understanding rather than just appearance matching.

Wei Zhou, Xiongwei Zhu, Zelin Xu et al.

Multi-task image generation models can be trained more effectively by treating each capability (T2I, local edit, global edit) as a separate velocity field and having the student learn to compose them on its own generated trajectories.

DanceOPD is a training framework that helps image generation models master multiple tasks—text-to-image, local editing, and global editing—without them interfering with each other. It uses a distillation approach where a student model learns from specialized 'capability fields' (velocity fields in flow-matching models), routing each image to the right expert for its task.

Danyal Rehman, Charlie B. Tan, Yoshua Bengio et al.

Autoregressive models can outperform flow-based approaches for molecular sampling by avoiding invertibility constraints and enabling better scalability—opening a new direction for physics-informed generative modeling.

This paper introduces Autoregressive Boltzmann Generators (ArBG), a new method for efficiently sampling molecular systems at equilibrium. Unlike previous approaches using normalizing flows, ArBG uses autoregressive models to generate molecular configurations faster and more accurately, with a large pre-trained model (Robin) achieving 60% better energy predictions on peptide systems.

Yu-Neng Wang, Sara Achour

Analog hardware can generate images 100x more efficiently than digital systems, but requires rethinking model design to match fixed physics-based dynamics rather than flexible neural networks.

This paper proposes Analog Interaction Systems (AIS), a framework for building generative models on analog hardware like coupled oscillators. The key innovation is bridging the gap between what neural networks can do and what analog physics naturally computes—using time-varying parameters and hidden states to improve expressivity while keeping energy costs 100x lower than digital approaches.

Andrew Bond, Zafer Doğan

Structured correlations in data can boost learning of weak features in GANs, but excessive correlation destabilizes training—there's a sweet spot determined by learning rates and noise.

This paper analyzes GAN training mathematically by studying how a linear generator learns data structure. The key innovation is handling realistic data with correlated features and class-dependent patterns—not just simple diagonal structure. The authors prove training converges to predictable equations and show that smart use of correlations can help weak features become learnable.

Sayak Dutta

Recurrent models can match Transformer efficiency by making forget gates content-aware (looking at stored memory) rather than memory-blind, enabling a mathematical solver that speeds up training while improving language understanding.

CARVE improves recurrent neural networks by fixing how they decide what to forget. Instead of gates that only see new incoming data, CARVE's gates look at what's already stored in memory before deciding what to erase. This single change fixes three architectural problems, enables faster training, and achieves better performance on language tasks while using less memory than competing approaches.

Ping Xiong, Thomas Schnake, Klaus-Robert Müller et al.

When explaining temporal graph models, you need to track information flowing through event-induced variables—not just embeddings—to capture how long-range dependencies actually work in the network.

This paper develops a new method to explain how Temporal Graph Neural Networks make predictions by tracking information flow through all components, not just embeddings. The approach uses a framework called Normalized Relevance Measure to systematically decompose complex temporal graph models and identify which events and interactions matter most for predictions.

Fabiana Fournier, Lior Limonad

You can add agentic AI capabilities to existing business processes by wrapping them with policy-governed agents at specific control points, rather than rebuilding the entire system.

This paper introduces a 'process harness'—a layer that adds AI reasoning to existing business workflows without replacing them. It uses three types of AI agents (for tasks, decisions, and flow control) that operate within policy guardrails, letting legacy systems maintain structural control while gaining adaptive intelligence at key decision points.

Haorui Ji, Weizhe Liu, Hongdong Li et al.

To build better image-to-3D systems, focus on using features designed for reconstruction rather than classification, and explicitly align 2D image information with 3D geometry during generation.

FLUX3D generates high-quality 3D Gaussian Splatting models from images by improving how 2D image features are converted into 3D representations and how the generation process aligns 2D and 3D data. It uses specialized techniques to preserve fine visual details that previous methods lost, resulting in better-looking 3D assets.

Jason Sulskis, Sathya Ravi

For PDE solvers, choose your spectral basis based on the operator's symmetry: real bases for elliptic PDEs, complex bases for time-dependent ones with phase content.

This paper compares two neural operators for solving PDEs: Fourier Neural Operators (FNO) using complex FFT and Hartley Neural Operators (HNO) using real Hartley transforms. Both are iso-parametric but use different spectral bases.

Zixuan Li, Haokun Lin, Yicheng Xiao et al.

Separating structure planning from appearance rendering in image generation improves prompt following for complex spatial and compositional requirements without needing intermediate outputs.

This paper improves text-to-image generation by separating structural planning from appearance rendering. IV-CoT uses two types of queries—structural and semantic—that work together in a single pass: structural queries create a latent visual plan (like an invisible sketch), then semantic queries render the final image based on that plan.

Zidu Liu, Florian Marquardt

LLMs can solve discrete design problems in quantum computing by evolving structured concepts rather than generating solutions from first principles—showing that domain-specific constraints and executable specifications make AI search more effective.

Researchers used large language models paired with structured algebraic rules to automatically discover new quantum error-correcting codes. Instead of designing codes from scratch, the system evolves mathematical specifications and programs that describe code families, finding competitive designs that work better than some existing approaches.

Reza Bayat, Ali Behrouz, Aaron Courville

You can improve language model efficiency by tapering MLP width across depth—allocating more capacity to early layers and less to later ones—a free performance gain that works across different architectures.

This paper shows that language models waste parameters by allocating them uniformly across layers. The authors propose Tapered Language Models, which gradually reduce the width of MLPs (the largest parameter-consuming components) from early to later layers. Across multiple architectures and scales, this simple change improves performance without extra cost.

Ruizhong Qiu, Yinglong Xia, Dongqi Fu et al.

Combining graph-based user co-engagement patterns with semantic tokenization creates more accurate user interest representations for generative recommendation systems at scale.

This paper presents G2Rec, a framework that improves generative recommendation systems by better organizing user behavior and item information. It combines graph-based user interaction patterns with semantic tokenization to help recommendation models understand what users want next, without needing labeled user interests.

Liang Su

For on-device AI agents that need to pause, branch, and resume execution frequently, capsules provide sub-millisecond state snapshots and 27x speedup on long contexts—a different optimization target than high-throughput LLM serving.

This paper introduces execution-state capsules, a checkpoint-restore mechanism for LLM serving on resource-constrained devices.

Jun He, Deying Yu

For production AI agents controlling infrastructure, enforce mutations through a separate broker that validates certificates at execution time, not just at decision time—this creates a mandatory checkpoint that agents cannot bypass.

This paper introduces Sovereign Execution Broker (SEB), a runtime enforcement system that sits between AI agents and infrastructure APIs to ensure mutations (changes) only happen when authorized by valid certificates.

Rostislav Makarov, Timo Gerkmann

You can reuse existing discriminative models (classifiers) for generative tasks by freezing them and training lightweight adapters, cutting the model footprint in half while keeping performance—useful when you already have trained classifiers lying around.

This paper shows how to repurpose a pre-trained speech classifier for generating speech by attaching a lightweight denoising network on top of it. Instead of training separate classifier and diffusion models, the authors freeze the classifier and train only a small adapter to guide generation, reducing memory and computation while maintaining high speech quality.

Siqi Miao, Shitij Govil, Jack P. Rodgers et al.

End-to-end transformers can match or beat graph neural networks on complex physics tasks while being dramatically faster and more memory-efficient—showing that careful architecture design beats multi-stage pipelines.

HEPTv2 is an end-to-end transformer model that reconstructs particle tracks from detector measurements at the Large Hadron Collider. It uses locality-sensitive hashing and sectorized decoding to achieve 98.6% accuracy while running 7-50x faster than competing approaches, making it practical for real-time physics experiments.

José A. Chávez

Timestep embeddings in diffusion models may be redundant—models can achieve competitive image quality without them by inferring noise scales directly from input corruption patterns.

This paper questions whether diffusion models actually need explicit timestep embeddings for denoising. The authors show theoretically and empirically that removing timestep information entirely doesn't significantly hurt performance on image generation tasks, and models can implicitly learn noise levels from corrupted inputs alone.

Ruida Wang, Rui Pan, Pengcheng Wang et al.

Diffusion-based language models outperform auto-regressive models for formal theorem proving by generating multiple tokens simultaneously, enabling better long-range coherence and error recovery—a paradigm shift for mathematical reasoning tasks.

This paper introduces Diffusion-Proof, the first framework using diffusion language models (which generate text by iteratively refining multiple tokens at once) for formal theorem proving. Unlike traditional auto-regressive models that predict one token at a time, diffusion models better maintain long-range coherence needed for complex proofs.

Zhaofeng Wu, Oliver Sieberling, Shawn Tan et al.

Not all transformer layers need the same width—narrowing middle layers while keeping early and late layers wide improves efficiency and performance, suggesting different layers have different computational roles.

This paper proposes Variable-Width Transformers, which use wider layers at the beginning and end of the network while narrowing middle layers. This non-uniform design outperforms standard transformers of the same size on language modeling, while reducing computation by 22% and memory usage by 15%.

Rishit Dagli, Donglai Xiang, Vismay Modi et al.

Sparse adaptive voxel structures with transformers can predict volumetric material properties 16× more efficiently than fixed-grid approaches, enabling high-fidelity physics simulation of complex 3D objects.

AdaVoMP predicts physical material properties (stiffness, elasticity, density) for 3D objects at high resolution. Instead of using fixed voxel grids, it uses a sparse adaptive structure with a transformer model, achieving 16× higher resolution than previous methods while using less computation. This makes 3D assets ready for realistic physics simulations.

Hongyuan Adam Lu, Z. L. Victor Wei, Qun Zhang et al.

Iterative refinement of latent states through looped computation offers a new scaling axis for world models—you can trade off depth for parameter efficiency without sacrificing simulation quality.

This paper introduces Looped World Models, which use parameter-shared transformer blocks that iteratively refine latent states instead of stacking deep layers. This approach achieves 100x parameter efficiency while maintaining faithful long-horizon simulation, and introduces adaptive computation that automatically adjusts depth based on prediction complexity.

Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin et al.

Fixed-point convergence provides a natural halting mechanism for iterative reasoning models, letting them use fewer steps on easy problems and more on hard ones—without explicit stopping signals.

This paper introduces FPRM, a looped Transformer that solves reasoning tasks by repeatedly applying the same layer until reaching a fixed point, automatically stopping when the model's internal state stabilizes. The approach addresses signal propagation problems in deep networks and adapts computation based on task difficulty.

Jisang Han, Seonghu Jeon, Jaewoo Jung et al.

Using 3D geometric reasoning as a shared foundation for both world prediction and action generation makes robot policies more accurate and efficient than 2D-based approaches, while requiring fewer parameters than large foundation models.

This paper introduces Geometric Action Model (GAM), a robot control system that uses a pretrained 3D geometry foundation model to understand both the physical world and predict robot actions.

Alper Yıldırım

Image classifiers internally separate identity (encoded in phase/sign) from magnitude information, similar to how human vision works, but different architectures expose this separation through different mechanisms—a mechanistic insight that could improve model design and interpretability.

This paper investigates whether trained image classifiers internally rely on phase information (like natural images do) by swapping phase and magnitude between images at different network layers.

Alper Yıldırım

Forecasting can be performed by passive optical diffraction rather than learned digital networks, achieving competitive or better results while suggesting the forecasting task itself may be fundamentally simpler than transformer-based approaches assume.

HAMON replaces learned digital layers in time-series forecasting with a passive optical system using diffraction and phase masks. Historical data is encoded onto an optical aperture, and future predictions emerge directly from light propagating through trainable phase masks—no digital sequence mixing needed.

Elias Lumer, Sahil Sen, Kevin Paul et al.

Spawning parallel subagents with independent tools and planning is more effective for complex long-context tasks than having a single agent handle everything—this architectural pattern improves coding performance from 71.75% to 81.36% on long documents.

This paper introduces Recursive Agent Harness (RAH), a pattern where AI agents spawn subagents to handle complex tasks in parallel rather than processing everything in a single call. Like how recursive language models break down reasoning into multiple model calls, RAH breaks down work into multiple agent instances, each with their own tools and planning capabilities.

Pedro Correa, Olivier Perrotin, Samir Sadok et al.

Phonetic information in speech representations is crucial for accurate 3D facial animation; discrete token-based representations can serve as an effective bridge between speech and facial motion synthesis.

This paper investigates which speech representations work best for animating 3D faces from audio. The researchers compare four types of speech encodings—self-supervised learning features, neural codec outputs, and ASR-based representations—and find that representations capturing phonetic information produce the most accurate facial animations.

Kyuil Lee, Dezhi Yu, Yongkang Huang

For symbolic music generation, autoregressive models with attention outperform VAEs and GANs at producing stylistically coherent Bach compositions, though vector quantization significantly improves VAE performance by preventing posterior collapse.

This paper compares three types of AI models for generating Bach-style piano music from symbolic notation: autoregressive models (which predict one note at a time), latent-variable models (which learn compressed representations), and adversarial models (which compete to fool each other).

Dachun Kai, Jiayao Lu, Yueyi Zhang et al.

Event cameras can significantly boost video super-resolution quality by providing high-temporal-resolution texture cues and motion information, improving detail recovery by up to 1.55 dB PSNR on texture-rich videos.

This paper introduces EvTexture++, a framework that uses event camera data to improve video super-resolution by enhancing texture details and temporal consistency. Event cameras capture high-frequency visual information that helps recover fine details in upscaled videos and reduce flickering artifacts caused by motion.

Cheng-Yu Yang, Shao-Yuan Lo, Yu-Lun Liu

Visual token importance changes as the model processes information deeper—tokens deemed unimportant early on may matter later, so recoverable routing outperforms permanent removal for vision-language tasks.

Vision-language models use thousands of visual tokens, making inference slow. Instead of permanently removing low-scoring tokens, this paper proposes Reroute: tokens can be temporarily skipped and re-evaluated later when they become important. The method works with existing token-reduction techniques and improves performance on grounding tasks without extra computational cost.

Yeongseo Jung, Jaehyeok Kim, Eunseo Jung et al.

Instead of truncating or summarizing conversations (which loses information), C-DIC stores revisable compression states organized by conversation threads, allowing the model to update and share information across turns efficiently without degrading dialogue quality.

This paper tackles the problem of long conversations becoming inefficient and error-prone in dialogue systems. The authors propose C-DIC, a method that compresses conversation history by organizing it into separate memory threads that can be updated and revised at each turn, rather than keeping the full history.

Songhao Wu, Ang Lv, Ruobing Xie et al.

Aligning router weights with the principal singular directions of experts improves MoE routing efficiency—a simple mathematical principle that scales from 1B to 11B parameter models.

This paper improves Mixture-of-Experts (MoE) models by redesigning how routers select which experts to use. The authors propose aligning each router with the most important direction of its expert using a mathematical technique called Manifold Power Iteration, which helps routers better match tokens to appropriate experts.

Danqi Zhuang, Jisui Huang, Xiaoyue Xi et al.

Using multiple periodic terminal distributions instead of one generic Gaussian helps diffusion models better understand and generate data with clear geometric structure, like points on manifolds or faces with consistent features.

This paper introduces PTL-Diffusion, a diffusion model that uses multiple structured terminal distributions instead of a single Gaussian. By embedding periodic structure directly into the forward noising process, the model better captures data that lies on low-dimensional manifolds, improving how well generated samples match the underlying geometric structure of the data.

Lennart Bastian, Samuel Leventhal, Mustafa Hajij et al.

TNOs separate the fixed topological rules governing information flow from learned transformations, enabling neural operators that respect physical geometry and conservation laws—useful for solving PDEs on irregular domains where standard approaches struggle.

This paper introduces Topological Neural Operators (TNOs), a framework for learning operators on complex geometric structures by explicitly modeling how information flows across different dimensional features (points, edges, faces, etc.) using topological calculus.

Wayne King, Zeyue Xue, Yuxuan Bian et al.

When building video world models, memory capacity matters more than compression, and the structure of how memory is accessed (like state-space recurrence) is as important as whether you use memory at all.

This paper systematically compares different memory mechanisms in video generation models that create multi-segment videos from text and camera actions.

Badr AlKhamissi, Johannes Mehrer, Lara Marinov et al.

A single spatial organization principle governs how the brain integrates different types of information (vision, sound, language), and this principle can be captured in AI models to predict and discover new functional brain regions.

Researchers built Topo-Omni, a neural network model that maps how the brain organizes visual, auditory, and language processing in a single unified spatial layout. By adding a spatial smoothness constraint to a pretrained foundation model, they discovered that nearby regions specialize in related tasks—just like real brains do.

Takuto Takahashi, Itsuki Nakayama, Takahiro Mitani et al.

GNNs fail on heterophilous graphs because they assume similar nodes connect; LCC fixes this by explicitly modeling how class labels relate at multiple hops, making it useful for real-world graphs where this assumption breaks down.

This paper tackles a key limitation of graph neural networks: they struggle when nodes with different labels are connected (heterophilous graphs). The authors propose Label Context Classifier (LCC), which captures higher-order patterns in how class labels connect in graphs by using multiple types of walks. LCC can be combined with any GNN to improve node classification performance.

Lizhi Yang, Junheng Li, Nehar Poddar et al.

By designing a compact command interface and distilling multiple specialist controllers, you can build a single humanoid controller that handles diverse real-world tasks while remaining simple enough for high-level planners to use.

HANDOFF is a whole-body control system for humanoid robots that uses a simple, intuitive command interface between task planning and physical control.

Akarsh Kumar, Phillip Isola

SMT decouples learning what to remember from how to update memory, enabling RNNs to train in parallel with stable gradients—potentially making RNNs competitive with Transformers for long-sequence tasks without requiring sequential computation.

This paper proposes Supervised Memory Training (SMT), a new way to train recurrent neural networks that avoids the sequential bottleneck of standard backpropagation through time. Instead of unrolling RNNs over long sequences, SMT trains a Transformer to learn what information to remember, then uses those memory labels to train the RNN in parallel.

Yutao Sun, Yanqi Zhang, Li Dong et al.

Reusing sparse attention routing decisions across layers dramatically reduces the computational cost of long-context inference without sacrificing quality—a practical win for deploying reasoning-heavy models with extended context windows.

This paper proposes cross-layer sparse attention (CLSA), a technique that speeds up long-context inference in large language models by computing which tokens to attend to once and reusing that decision across all decoder layers. By sharing both the key-value cache and routing decisions, it achieves significant speedups (up to 7.6x) while maintaining accuracy on long-context tasks.

Ammar Hoori, Yuichi Motai

Splitting datasets spatially and training specialized neural networks in parallel using evolutionary algorithms can outperform traditional gradient-based training while being faster and more scalable.

This paper proposes two new neural network training methods (MC-PSO and MC-APSO) that split datasets into spatial subsets and train small specialized neural networks on each subset using particle swarm optimization. During prediction, only relevant networks contribute to the final answer, improving both accuracy and speed compared to existing methods.

Gandhimathi Padmanaban, Fred Feng

For real-world computer vision deployment, a two-stage pipeline with confidence-based abstention (refusing to predict when uncertain) is more reliable than forcing predictions—the model's uncertainty directly predicted where it would fail on new data.

Researchers built an open-source tool that automatically identifies vehicle types from roadway video to help understand cyclist safety in crashes. The system uses two AI models in sequence: first detecting vehicles in video frames, then classifying them into six specific body types (car, SUV, pickup truck, etc.).

Josef Bengtson, Yaroslava Lochman, Fredrik Kahl

This method lets you edit one image of an object and automatically propagate those changes—including major shape changes—consistently across all other camera angles without retraining.

GeM-NR enables consistent editing of multi-view images when geometry changes dramatically—like reshaping objects or altering scene structure. Unlike prior methods that only handle appearance changes, it works with any image editor (FLUX, Qwen, BrushNet) by aligning 3D geometry between edited and unedited views, then intelligently projecting and refining edits across viewpoints.

Hongyu Zhou, Zorah Lähner

Standard 3DGS conflates appearance and geometry, hurting both. Adding a geometry-specific opacity parameter lets you get better rendering quality AND more accurate 3D surfaces, especially for complex scenes with transparent objects.

This paper identifies a fundamental limitation in 3D Gaussian Splatting (3DGS)—it struggles to represent both realistic appearance and accurate geometry simultaneously. The authors propose a simple fix: adding a separate opacity parameter for geometry to each Gaussian splat, which decouples how the model handles texture rendering versus surface reconstruction.

Amil Dravid, Yasaman Bahri, Alexei A. Efros et al.

Neural networks don't just get better at tasks with scale—their internal neuron populations reorganize predictably, with interpretable neurons becoming more specialized while others remain general, offering a new lens for understanding how model size shapes network structure.

This paper reveals that as neural networks grow larger, certain neurons called Rosetta Neurons become more selective and specialized, following predictable scaling patterns. While these interpretable neurons increase in absolute number, they shrink as a percentage of total neurons, and the remaining neurons become less selective—a phenomenon the authors call Neuron Polarization.

Lianghuan Huang, Yihao Li, Saeed Salehi et al.

Vision Transformers struggle with binding (knowing which features go together), and this limitation explains why they fail at tasks involving feature-sharing or occluded objects—making binding a measurable and critical component of visual understanding.

This paper formalizes the 'binding problem'—how AI models know which visual features belong to the same object—using information theory. The authors develop a probing method to measure binding information in Vision Transformers and show that binding is crucial for understanding scenes with multiple objects, especially when objects share features or overlap.

Haowen Hou, Zhen Huang, Zheming Liang et al.

Video MLLMs can be dramatically more efficient by encoding frames predictively: send full frames only when needed, use compact change descriptions otherwise. This cuts token usage to 1/7 while improving accuracy and reducing latency from 9+ seconds to 1.6 seconds.

AdaCodec is a new way to encode videos for AI models that intelligently decides when to send full frames versus compact descriptions of changes.

Junxia Cui, Haotian Ye, Runchu Tian et al.

Diffusion language models can now use speculative decoding—a proven speedup technique from autoregressive models—by using a simple masking strategy that preserves valid token context during verification.

This paper introduces SimSD, a technique that speeds up diffusion language models by enabling them to verify multiple predicted tokens at once, similar to how autoregressive models work. The key innovation is a masking strategy that gives diffusion models the right context to check draft predictions efficiently, achieving up to 7.46x faster inference without sacrificing quality.

Hidir Yesiltepe, Jiazhen Hu, Tuna Han Salih Meral et al.

Low-rank KV cache compression works in video diffusion not because attention is inherently low-rank, but because the model learns to use whatever rank capacity is available—this insight could improve efficiency of long-context generation across domains.

This paper introduces VideoMLA, a technique that compresses the key-value cache in video diffusion models by using shared low-rank representations instead of per-head storage.

Qinpei Luo, Ruichun Ma, Xinyu Zhang et al.

Representation design matters more than model size: a semantically-grounded code format for PCB schematics lets smaller LLMs outperform larger general-purpose models on hardware design tasks.

SchGen is the first AI system that generates PCB schematics from natural language descriptions. The key innovation is a new code-based representation that focuses on semantic relationships (like component connections) rather than geometry, making it easier for language models to learn. The authors also built a large dataset by converting open-source hardware designs into this representation.

Chong Bao, Shichen Liu, Lijun Yu et al.

A single pretrained model can now generate realistic digital humans across all modalities (speech, movement, appearance) by treating them as a unified problem rather than separate tasks, making it practical to build avatar systems without specialized sub-models.

Archon is a unified AI model that generates digital humans across multiple modalities—text, audio, motion, and video—all from a single system. It uses a novel video compression technique to handle high-resolution talking videos efficiently, and a step-by-step reasoning approach that switches between modalities to improve output quality.

Sayan Paul, Sourav Ghosh, Siddharth Katageri et al.

For building 3D city simulations from images, this method scales efficiently by partitioning large scenes and processing them separately, avoiding the computational explosion and geometry problems that plague existing approaches.

City-Mesh3R reconstructs large-scale city 3D meshes directly from photos using a divide-and-conquer approach. Instead of processing entire cities at once, it breaks scenes into manageable chunks, reconstructs each independently, then stitches them together—producing clean, simulation-ready meshes without the incomplete geometry or computational bottlenecks of existing methods.

Abhilash Durgam, Nyle Siddiqui, Jeffrey A. Chan-Santiago et al.

State space models with causal (one-directional) processing are more efficient than attention-based models for streaming EEG analysis, and specialized self-supervised training can teach them to remember important brain events separated by long time gaps.

CaMBRAIN is a new AI model for analyzing brain activity from EEG signals in real-time. Unlike existing models that struggle with long recordings, it uses a causal state space architecture that processes signals sequentially without the computational overhead of attention mechanisms, enabling continuous analysis of hours-long EEG data while running 10x faster.

Shihao Wang, Shilong Liu, Yuanguo Kuang et al.

Decoding bounding boxes as complete geometric units instead of individual tokens dramatically speeds up inference while maintaining or improving localization accuracy.

LocateAnything replaces slow token-by-token box decoding with Parallel Box Decoding, which generates entire bounding boxes at once. Combined with a 138-million-sample dataset, this approach makes visual grounding and detection faster while improving accuracy on standard benchmarks.

Yanbei Chen, Hanxian Huang, Ernie Chang et al.

MoE isn't just for giant models—on mobile devices, moderate sparsity with shared experts is both memory and compute-optimal, letting you get better performance with fewer active parameters than dense models.

MobileMoE brings Mixture-of-Experts (MoE) architecture to phones and edge devices by optimizing it for memory and compute constraints. The models use 0.3-0.9B active parameters but achieve better performance than larger dense models, running 2-4× faster on real smartphones while using less memory.

Nithesh Chandher Karthikeyan, Jonas Unger, Gabriel Eilertsen

You can guide diffusion models to generate specific images by using learned representations as conditioning signals, avoiding the need for expensive annotated datasets while maintaining smooth, interpretable control.

This paper shows how to control image generation in diffusion models by conditioning them on representations from self-supervised models instead of requiring text or semantic annotations. The approach discovers interpretable directions in the representation space that let you smoothly control what gets generated.

Shangding Gu

Agent performance depends equally on system design (memory, routing, verification) as on model capability; evaluating agents requires measuring trajectory quality and system hygiene, not just final outcomes.

This paper argues that building better AI agents requires focusing on the system architecture around language models, not just making the models bigger. It introduces the concept of 'scaling the harness'—designing the memory, tool-use, verification, and orchestration layers that turn a model into a working agent—and proposes benchmarks to measure agent quality beyond just task success.

Shuhong Zheng, Aashish Kumar Misraa, Yu-Teng Li et al.

Jointly encoding text and images in MLLMs before conditioning diffusion models preserves subject identity better than separate encoding, while a multi-stage denoising strategy balances semantic instruction-following with fine-detail preservation.

This paper improves subject-driven image generation by using multimodal large language models (MLLMs) to jointly understand text and reference images together, rather than separately. The approach adds a VAE-based identity module and a novel aggregation technique to balance semantic understanding with preserving the subject's identity, reducing unwanted copy-paste artifacts.

Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie et al.

A plug-in architecture for multimodal continual learning lets researchers test new training strategies without rewriting the base model code, making MLLM research faster and more reproducible.

Prism is a software framework that makes it easier to develop and test new methods for continuously training multimodal AI models on new tasks. Instead of modifying the core model code each time, researchers can add new strategies as plug-in modules, reducing engineering overhead and enabling fair comparisons between different approaches.

Sanghyun Lee, Chunsan Hong, Seungryong Kim et al.

Selectively looping transformer layers in masked diffusion models improves both training efficiency and reasoning capability—you can match performance with far fewer computations, or trade compute for better results.

This paper introduces LoopMDM, a technique that reuses early-middle transformer layers in masked diffusion models by looping them during training and inference. The approach achieves better training efficiency (3.3× fewer FLOPs) and stronger reasoning performance than standard models, while enabling flexible compute scaling at inference time without adding parameters.

Sangyun Lee, Sean McLeish, Tom Goldstein et al.

Language models can improve long-context reasoning by periodically consolidating recent information into fast weights during offline 'sleep' phases, trading inference latency for better performance on reasoning-heavy tasks.

This paper proposes a sleep-like mechanism for language models that periodically consolidates recent context into persistent memory before clearing the cache. During 'sleep,' the model performs offline passes to update fast weights in state-space blocks, shifting computation away from real-time inference.

Maoyang Xiang, Bo Wang, Tao Luo

Power-of-Two quantization with orthogonal residual projection lets you run large models on edge hardware with minimal accuracy loss and no multiplier circuits—calibration takes ~15 minutes instead of hours.

OrpQuant enables efficient deployment of large AI models on edge devices by using a novel quantization method that replaces expensive multiply operations with simple bit-shifts. The approach uses geometric projection to maintain accuracy even at ultra-low bit widths (3-4 bits), and can calibrate models 10x faster than existing methods.

Wei Song, Tianhang Wang, Yitong Chen et al.

Quantizing image channels instead of patches improves codebook efficiency and enables a more intuitive generation process that mirrors human artistic creation, achieving strong text-to-image results.

This paper introduces Channel-wise Vector Quantization (CVQ), a new way to convert images into discrete tokens by quantizing color channels instead of spatial patches. It enables a new image generation model (CAR) that builds images progressively—first sketching overall structure, then adding fine details—similar to how artists work.

Ali Hatamizadeh, Yejin Choi, Jan Kautz

Decoupling erase and write operations in linear attention with separate gates improves language model performance, especially on long-context tasks, while maintaining constant-memory decoding.

This paper improves linear attention mechanisms by separating the control of what to forget from what to remember in compressed memory. Instead of using a single gate to control both erasing old information and writing new information, Gated DeltaNet-2 uses separate channel-wise gates for each operation, making memory updates more flexible and efficient.

Amir Mousavi, Mohammad Sadegh Sirjani, Erfan Nourbakhsh et al.

Mamba's linear-complexity architecture enables real-time cognitive load monitoring from noisy eye-tracking signals on wearable devices—a practical alternative to Transformers for temporal sensor data with frequent gaps.

MambaGaze uses a bidirectional Mamba neural network to assess cognitive load from eye-tracking data in real-time. It handles missing data from eye blinks and tracking failures by explicitly encoding uncertainty, and runs efficiently on edge devices like smartglasses for applications like driver monitoring.

Samson Gourevitch, Yazid Janati, Dario Shariatian et al.

Discrete diffusion models have a hidden training-inference mismatch: the standard objective doesn't match what's actually needed for sampling. Using the correct "leave-one-out" parameterization and an absorbing-state reformulation improves generation quality without retraining.

This paper fixes a fundamental mismatch in how Uniform Diffusion Models are trained versus used for generation. The authors show that standard training doesn't actually optimize what the model uses during sampling, and they provide mathematical conversions to align these.

William Smits

Decision trees can improve accuracy by explicitly handling boundary cases through locally-computed uncertainty zones—instances near splits get soft predictions and uncertainty flags instead of hard classifications, helping downstream applications make better decisions.

This paper introduces ternary decision trees that add uncertainty zones around split thresholds, allowing predictions near decision boundaries to blend outputs from both child subtrees and flag uncertain cases.

Yan Yang, Bin Gao, Ya-xiang Yuan

Optimization on manifold intersections becomes tractable when intrinsic transversality holds; a geometric algorithm can efficiently solve these problems by maintaining feasibility on one manifold while steering toward the intersection.

This paper tackles optimization problems where the solution must lie on the intersection of two geometric surfaces (manifolds). The authors prove that two key geometric properties are equivalent, enabling efficient projection onto the intersection.

Mansoor Ahmed, Sujin Lee, Umar Khayaz et al.

Combining evolutionary knowledge from language models with 3D structural constraints solves vocabulary collapse in antibody design, achieving 16% better sequence accuracy and 2.3x more amino acid diversity than structure-only methods.

EvoStruct fixes a critical problem in AI-designed antibodies: neural networks trained on 3D structures alone forget important amino acid patterns from evolution. The method combines a pre-trained protein language model (which knows evolutionary patterns) with structural information, using a special adapter to merge both sources of knowledge.

Tilman Tröster, David Mirkovic, Veronika Oehl et al.

Matching a model's architectural symmetries to the actual symmetries present in your data—not just the underlying physics—significantly improves performance and data efficiency.

Velocityformer is a specialized neural network that reconstructs galaxy velocities from survey data to improve cosmological measurements. By designing the model to match the asymmetric structure of real observations (where one direction—the line of sight—is special), it achieves 35% better accuracy than traditional methods and works well even with very limited training data.

Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti et al.

DashAttention enables efficient long-context processing by combining adaptive sparse selection with differentiable training, outperforming fixed-sparsity methods while maintaining gradient flow through both attention stages.

DashAttention improves how language models handle long documents by using a smarter two-stage attention mechanism. Instead of always selecting the same number of relevant tokens, it adaptively picks different amounts based on what each query needs, while keeping the entire process trainable. This achieves full-attention quality with 75% fewer computations.

Xuying Ning, Katherine Tieu, Dongqi Fu et al.

Code is becoming the primary substrate for building reliable, verifiable AI agents. Understanding code as agent harness—the infrastructure layer—is essential for building systems that can plan, remember, use tools, and coordinate across multiple agents.

This survey examines how code serves as the operational foundation for AI agents—not just as output, but as the infrastructure that enables agents to reason, act, model environments, and verify their own behavior.

Kunqi Xu, Jitao Li, Jianglong Ye et al.

By explicitly modeling object state changes as a learnable manifold, WorldString provides a unified way to represent how objects respond to actions—bridging the gap between perception and control for physical world models.

WorldString is a neural architecture that learns to represent how real-world objects change state over time by processing point clouds or video data. It creates a digital twin of objects that captures their actionable properties, serving as a building block for world models that can predict and interact with the physical world.

Miguel Farinha, Ronald Clark

By conditioning on intrinsic image properties (albedo and shading) extracted from both photos and 3D renders, you can achieve photorealistic relighting with full PBR lighting control while staying fast enough for practical use.

PIXLRelight is a fast neural relighting method that lets you change lighting in photos using physically-based rendering controls. It decomposes images into intrinsic components (albedo, shading, residuals) and uses these to condition a transformer model, enabling realistic lighting adjustments in under 0.1 seconds per image without per-image optimization.

Songsong Yu, Yuxin Chen, Ying Shan et al.

Using segmentation as a generative training task bridges the gap between visual understanding and generation in multimodal models, improving both capabilities simultaneously rather than training them separately.

This paper shows how to train unified multimodal models (that do both image understanding and generation) more effectively by using image segmentation as a training task. Instead of training understanding and generation separately, the authors use segmentation to align both capabilities, improving the model's ability to understand images and generate them accurately.

Ellwil Sharma, Arastu Sharma

Sparse mixture-of-experts routing can solve the problem of conflicting physics domains in foundation models by automatically routing different physics problems to specialized experts while maintaining shared knowledge for universal principles.

This paper tackles negative transfer in multi-physics AI models—where training on different physics problems simultaneously hurts performance. The authors propose Shodh-MoE, which uses sparse expert routing to let different parts of the model specialize in different physics regimes (like fluid dynamics vs. porous media flows) while sharing knowledge where it helps.

Alan Z. Song, Yinjie Chen, Mu Nan et al.

You can build efficient vision transformers by routing all patch interactions through a small set of learned core tokens instead of using all-to-all attention, achieving linear complexity without sacrificing performance.

This paper proposes VECA, a vision transformer that replaces quadratic all-to-all attention with linear-time attention using learned "core" tokens as communication hubs. Instead of every patch attending to every other patch, all patches only interact through a small set of learned cores, reducing computation from O(N²) to O(N) while maintaining competitive accuracy on vision tasks.