Papers

Tate Berenbaum, Muthaiah Venkatachalam

Pipeline parallelism with pre-compiled shards lets you run 70B models on clusters of consumer AI PCs at interactive speeds by splitting computation across machines and optimizing GPU utilization through careful kernel fusion and speculative decoding.

This paper shows how to run large language models (like Llama 70B) across multiple Intel AI PCs by splitting the model into layers and running each layer on a different machine. Using OpenVINO optimization, speculative decoding, and request interleaving, they achieve near-optimal speed while serving multiple users simultaneously on modest hardware.

Reza Bayat, Ali Behrouz, Vahab Mirrokni et al.

Gradually expanding memory capacity during sequence processing—rather than using static memory—is a simple, cost-free improvement that works across different memory-based architectures and shows larger gains on longer contexts.

This paper introduces Proteus, a technique that improves how neural networks handle long text by gradually expanding memory capacity as more context arrives.

Hanfeng Lu, Tianyu Feng, Suyi Li et al.

By overlapping independent computation phases and sharing GPU memory intelligently, you can train vision-language models 1.2–2.2× faster without needing more hardware or changing your RL algorithm.

Rollplex is a GPU runtime that speeds up vision-language model training by overlapping different computational phases.

Hao Yan, Lisa Pilgram, Dan Liu et al.

You can now generate realistic synthetic tabular health data from multiple heterogeneous sources by standardizing them statistically first, then using diffusion transformers to learn and reproduce their patterns.

This paper presents a method for generating synthetic health data from multiple different database tables with varying structures. It works in two stages: first converting diverse tables into a standardized statistical format, then using a diffusion transformer to learn patterns and generate new synthetic tables.

Abhishek Shukla, Ankur Sinha, Faiz Hamid

By formulating architecture search as a linear program using gradient and curvature information, LP-NAS finds better neural network designs 2-3x faster than standard differentiable NAS methods while achieving higher accuracy.

This paper proposes LP-NAS, a neural architecture search method that uses linear programming to find better network designs faster. Instead of randomly exploring architectures, LP-NAS uses mathematical optimization principles to guide the search, resulting in architectures that generalize better and are found more quickly than existing methods like DARTS.

David Chushig-Muzo, María Ángeles Rodríguez de Cara, Eva Milara et al.

Self-organizing maps can encode tabular data more effectively than simpler dimensionality reduction by capturing feature relationships alongside values, improving both predictive performance and model interpretability.

TabSOM converts tabular data into images using self-organizing maps, preserving both feature values and relationships between features. Unlike existing methods that only encode individual feature values, TabSOM captures feature interactions as spatial patterns, enabling vision models to achieve better performance while remaining interpretable.

Sabin Roman, Ljupco Todorovski, Saso Dzeroski

SORT shifts equation discovery from brittle library selection to basis design: by learning sparse coefficients in well-chosen orthogonal bases, it provides a more stable intermediate representation that gracefully degrades under noise and sampling sparsity.

SORT is a machine learning technique that learns compact mathematical representations of dynamical systems from noisy, irregularly sampled data by fitting sparse coefficients in orthogonal basis expansions.

Daniel Perkins, John Squires, Janou Milligan et al.

Instead of training separate models for each domain, you can use an MLLM router to dynamically select which vision backbone handles each image, getting both better generalization and easier updates without retraining.

ARMDIL uses a multimodal language model to intelligently route images to the best-suited vision model (CNNs, self-supervised learners, or vision-language models) within an ensemble.

Van Khoa Nguyen, Alexandros Kalousis

By reversing the physics concept of spontaneous symmetry breaking, this diffusion model generates complete crystal structures with proper global symmetries—a significant improvement over methods that only generate partial specifications.

This paper proposes a new method for generating crystal structures by using a diffusion-based model that starts from low-symmetry configurations and gradually breaks symmetries to create complete crystal specifications.

Andy Yang, Blerta Veseli, Corentin Barloy et al.

Transformers' ability to generalize to longer sequences depends on specific algebraic properties of the language being learned—properties that classical finite algebra misses but can be captured by extending decomposition theory to infinite groups.

This paper characterizes which regular languages transformers can generalize to longer sequences than they've seen during training. The authors develop new algebraic theory extending classical decomposition methods to handle transformers' unbounded counting abilities, providing a polynomial-time algorithm to predict length generalization on any regular language.

Aimilios Hadjiliasi, Louis Nisiotis

Small language models can efficiently run agent cognition (thinking and memory) on edge devices like Jetson boards, enabling virtual agents to operate independently in real-time without cloud latency.

This paper explores how small language models (SLMs) running on edge devices can power the cognitive processes of virtual agents in immersive worlds.

Paul Savala

Predicting course enrollment alongside grades significantly improves academic performance forecasting—jointly modeling what students take and how they'll perform is more accurate than predicting grades alone.

This paper presents TRACE, a transformer-based model that jointly predicts which courses students will take and their grades in those courses for upcoming semesters. Unlike traditional approaches that treat student history as a simple sequence, TRACE captures how courses taken concurrently within a semester affect performance, reducing prediction error by nearly 50% compared to grade-only models.

Chuyue Li, Jinpeng Yu, Haozhe Wang et al.

Agents can now learn from and generate high-quality videos by working with structured knowledge graph representations instead of raw pixels, improving video generation quality by 20.7% over existing methods.

AVA-Encoder learns video representations as knowledge graphs that agents can reason about and edit. It converts videos into structured text and asset layers, then reconstructs videos from these representations. A natural-language feedback loop optimizes the encoding, enabling agents to work with cinematic-quality videos more effectively.

Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou et al.

Combining neighborhood attention mechanisms with uncertainty-weighted loss functions enables accurate segmentation of tiny, low-contrast anatomical structures even with limited training data—a practical approach for clinical imaging tasks where manual annotation is expensive and ambiguous.

This paper presents NA-UNETR, a transformer-based deep learning model for segmenting the Left Anterior Descending artery in CT scans.

Eric A. F. Reinhardt, Adam J. Hauser

Softmax attention has an exact quantum mechanical interpretation where attention scores become measurement statistics and the softmax function emerges naturally from quantum probability rules, opening a path to quantum implementations of transformers.

This paper shows how softmax attention in transformers can be exactly implemented using quantum computing. The researchers map each component of attention—from computing scores to aggregating values—to quantum operations like measurements and rotations. They prove this works mathematically and verify the core logic in formal proof software.

Davide Rinaldi, Luciano Serafini

If you're building neurosymbolic systems that need to reason about ordered or connected data, sLTN lets you write logical rules about structure (like "event A happens before event B") and train them end-to-end with neural networks.

sLTN extends Logic Tensor Networks to handle structured data like sequences and graphs by treating structural dimensions (time steps, positions, nodes) as first-class elements in the logical language. This lets you express temporal and relational constraints directly in logic while keeping everything differentiable for neural learning.

Alban Puech, Matteo Mazzonelli, Tamara R. Govindasamy et al.

Neural networks can solve critical power grid problems 30-85x faster than traditional solvers while staying physically consistent—opening the door to AI-powered grid analysis at scale.

GENCO is a neural network solver that handles power flow, optimal power flow, and state estimation for electrical grids in a single unified model. It matches or beats classical solvers like Newton-Raphson and IPOPT in speed and accuracy while maintaining physical consistency, and comes with an open-source framework and large datasets for reproducible research.

Abraham Gonzalez, Raghav Gupta, Akanksha Jain et al.

Agentic AI can discover better microarchitecture designs than humans by breaking large search spaces into manageable pieces and embedding hardware constraints directly into the optimization loop.

ArchAgent v2 automatically designs data prefetching policies for computer processors using evolutionary search, beating hand-designed solutions in a competition. It introduces cascaded evolution (optimizing cache levels sequentially) and hardware-realizability feedback to handle the massive design space, achieving 3.8% performance improvement over baseline.

Sam Levang, Fran Bartolic, Ty Dickinson et al.

You can now configure forecast timesteps after training instead of before, letting one model handle both detailed short-range and stable long-range weather predictions without retraining.

GEM-3 is a weather forecasting model that solves a key problem: existing models must choose between short timesteps (detailed but error-prone) or long timesteps (stable but missing short-term details). This model lets you pick the timestep at inference time with a single trained model, and training on mixed timesteps makes forecasts more stable across longer periods.

Elad Yoshai, Natan T. Shaked

Per-feature gating based on distribution distance enables better control in unpaired image translation, letting you preserve important structures while achieving realistic style changes without retraining.

PRISM is a new method for changing images from one style to another (like turning day photos into night) without paired training examples. Instead of using a single global control value, it learns a per-feature gate based on how far each image feature is from the target style, allowing precise control over what changes and what stays the same.

Hao Ding, Daniel Semchin, Paul M. Thompson et al.

Predicting a surface's intrinsic geometric properties (metric tensors) in continuous time, rather than directly predicting vertex positions or embeddings, produces more accurate and geometrically valid brain structure forecasts for clinical applications.

This paper presents MT-GNN, a graph neural network that predicts how brain structures will change over time by learning their intrinsic geometry (metric tensors) rather than directly predicting shape. Given a patient's past brain scans, the model forecasts the mathematical properties that define a surface's shape at future timepoints, then reconstructs the actual 3D surface.

Purbesh Mitra, Sennur Ulukus

Breaking complex reasoning into multiple fresh inference passes with shared artifacts lets LLMs catch and correct mistakes earlier, improving accuracy on multi-step tasks like counting, ordering, and multi-hop reasoning.

This paper proposes Chained Recursive Language Models (Chained RLM), a system where an LLM is called multiple times in sequence to solve complex reasoning tasks. Instead of trying to do everything in one long response, each call gets the original problem plus a summary of previous work, allowing the model to inspect and fix earlier mistakes before moving forward.

Arunava Majumder, Marius Krumm, Hendrik Poulsen Nautrup et al.

Shallow quantum generative models become strictly more powerful when augmented with classical randomness—a weak resource that enables long-range correlations impossible for unitary-only circuits at fixed depth.

This paper proves that adding shared classical randomness to shallow quantum circuits can generate probability distributions that require much deeper purely quantum circuits to replicate. The authors show this separation is achievable with simple local operations controlled by a single random bit, and demonstrate how measurement-based quantum computation naturally implements this approach.

Yang Yang, Qinyu Zhao, Mouxiang Chen et al.

By sharing backbone parameters across parallel branches with task-specific computation allocation, you can improve multimodal model performance without increasing model size or inference latency.

ParVL introduces a framework for scaling multimodal AI models by running multiple parallel vision and language processing branches that share the same core parameters. Instead of making models bigger or slower, it reuses existing components more efficiently and lets different tasks use different amounts of vision vs.

Junhao Chen, Mingjin Chen, Jingjia Mao et al.

Music tokenization design is more important than model size for text-to-music generation—a small model with the right token representation outperforms massive models with poor representations, challenging the field's scaling assumptions.

This paper investigates how music tokenization—the way music is converted into discrete symbols for language models—affects text-to-music generation quality. By fixing model size, data, and training approach while swapping only the tokenization scheme, researchers found that representation choice matters far more than model scale.

Christopher Schröder, Lukas Gienapp, Ferdinand Schlatt et al.

ALiBi positional encoding silently fails on long sequences due to numerical precision issues, degrading retrieval performance; using log-scaled distances instead of linear scaling provides the most reliable fix.

This paper discovers that ALiBi positional encoding—a popular method for handling long sequences in transformers—has a critical flaw: its linear bias scaling causes floating-point underflow, making many attention weights zero and blinding attention heads. The authors analyze this problem, test fixes, and show it hurts token retrieval tasks while barely affecting standard benchmarks.

Cencen Liu, Wen Yin, Dongyang Zhang et al.

Separating degradation information from scene content using orthogonal feature subspaces significantly improves all-in-one image restoration quality, achieving state-of-the-art results across multiple degradation types.

This paper tackles all-in-one image restoration—fixing multiple types of image damage (blur, noise, rain, etc.) in one model. The key problem is that existing methods mix degradation information with image content in a shared space, causing confusion.

Zechen Zhang, Rui Peng Li, Yousef Saad

GNNs can learn better preconditioners for sparse linear systems by adopting classical multigrid hierarchies as structural priors, but multilevel structure doesn't always outperform simpler single-level learned approaches—the tradeoff depends on the problem type.

This paper proposes Graph Neural Multilevel Preconditioners (GMP), which combines classical algebraic multigrid structure with graph neural networks to speed up solving large sparse linear systems. The method learns how to smooth, restrict, and interpolate data across multiple levels while maintaining compatibility with standard iterative solvers, tested on over 800 real sparse matrices.

Tom Saliencro, Rohan Desai, Priya Nair et al.

You can route tokens to variable numbers of experts based on per-token confidence signals from the router itself, improving efficiency and performance without adding parameters or complexity.

This paper proposes CARE, a smarter way to route tokens through expert networks in fine-tuned language models. Instead of sending every token to the same fixed number of experts, CARE uses the model's own uncertainty signals to decide how many experts each token needs—easy tokens get fewer experts, hard tokens get more.

Solomon Micheal Serunjogi, Rachmad Vidya Wicaksana Putra, Ayat Taha et al.

Using spatial modes instead of wavelengths in photonic accelerators enables simpler, more efficient transformer inference with a single laser source and no spectral filtering, making photonic acceleration more practical for real-world deployment.

MDTransformer is a photonic hardware accelerator for transformer models that uses spatial light modes instead of wavelengths to perform matrix operations.

Neta Shaul, Chao Liu, Arash Vahdat et al.

PDD enables fast video/image generation (4-8 steps instead of 50+) with simpler training than current methods, while actually improving output diversity—making it practical for scaling generative models to production.

This paper introduces Parallel Decoding Distillation (PDD), a method to speed up image and video generation from diffusion models by predicting multiple denoising steps at once instead of one at a time.

Francisco Mena, Dino Ienco, Roberto Interdonato et al.

Co-learning between modalities is more robust than fusion-based approaches when dealing with unpredictable missing modalities at inference time, with different strategies optimal for mild versus extreme missing conditions.

This paper tackles multi-modal classification when some data sources are unavailable at inference time—a common real-world problem from sensor failures or privacy constraints. Instead of designing better fusion methods, the authors use co-learning where modalities teach each other.

Wenhao Li, Xueying Jiang, Quanhao Qian et al.

VLMs can be made 3D-aware by injecting learned geometric representations from video, enabling strong performance on spatial reasoning tasks while remaining RGB-only—no special 3D sensors or data needed.

This paper enhances vision-language models to better understand 3D spatial information by adding two types of geometric representations learned from RGB videos: implicit geometry tokens that capture high-level 3D structure, and explicit geometry tokens that encode detailed geometric details.

Sophia Tang, Pranam Chatterjee

EFMs enable flow-based models to generate variable-length outputs by decomposing generation into expandable state spaces, making output size a learnable parameter rather than a fixed constraint.

This paper introduces Expanding Generative Flows (EFlows) and Expanding Flow Maps (EFMs), a new framework for flow-based generative models that can generate outputs of varying sizes.

Vedant Shah, Onkar Susladkar, Tushar Prakash et al.

Using structured graphs to represent object relationships provides a more scalable and precise way to control video generation than trajectory drawing or text prompts, especially for complex multi-object scenes.

GraphVid enables precise control over multi-object interactions in video generation by using structured interaction graphs instead of text or pixel-level motion inputs. The method outperforms existing motion-control approaches while using less training data, and the authors release GraphVid-Bench, a new dataset with relational annotations for interaction-aware video generation.

Aaron Feller, Kris Deibler, Maxim Secor

Modeling molecules as conformational ensembles rather than single structures substantially improves property prediction—pretraining on ensemble data is critical, as the model fails from scratch but reaches strong performance when pretrained.

This paper introduces EnsembleEGNN, a neural network model that predicts molecular properties by learning from multiple 3D structures (conformations) of cyclic peptides simultaneously, rather than just one.

Amirhossein Sadr, Nima Soltani, Vahideh Moghtadaiee et al.

By pairing KANs with Petrov-Galerkin weak formulations instead of strong-form residuals, you get better numerical stability, lower computational cost, and broader applicability to real physics problems—without sacrificing accuracy.

This paper introduces PG-KINN, a physics-informed neural network that combines Kolmogorov-Arnold Networks (KANs) with a Petrov-Galerkin mathematical formulation to solve differential equations.

Rahul Sajnani, Yulia Gryaditskaya, Radomír Měch et al.

Appearance pointers provide a modality-agnostic way to add regional control to existing Diffusion Transformers, letting you specify exactly where text descriptions or image references should apply in the generated output without expensive retraining.

This paper introduces appearance pointers, a new technique for controlling where and how text and image inputs influence image generation in Diffusion Transformers.

Ioannis Papageorgiou, Srinivas Nomula, Ayalvadi Ganesh et al.

Combining O(log K) binary classifiers to solve K-class problems has fundamental limits that depend on data dimensionality and noise—understanding these limits helps design efficient distributed classification systems.

This paper studies how to build a multiclass classifier by combining many simple binary classifiers in a distributed way. The authors analyze the fundamental performance limits when using hyperplane-based binary classifiers on Gaussian data, deriving theoretical bounds across different scenarios and validating them with experiments.

Chirag Vashist, Ke Li

Single-step generative models can match multi-step diffusion models' quality without gradual transformations—simplicity in training objectives and architecture doesn't sacrifice performance.

This paper challenges the assumption that gradual noise-to-data transformations are necessary for good generative models. The authors build a minimalist single-step generative model using Implicit Maximum Likelihood Estimation (IMLE) and a simple convolutional network, achieving competitive results (FID 2.56 on ImageNet 256) without iterative denoising or transformers.

Chen Ziheng

You can now build neural networks on curved spaces using reusable, theoretically-grounded components instead of ad-hoc manifold-specific solutions, with practical efficiency improvements for important geometric representations.

This thesis develops a unified framework for building deep neural networks that work with data on curved mathematical spaces (manifolds) rather than flat Euclidean space.

Ziyang Cai, Xingyu Zhu, Yihe Dong et al.

Adding recurrence to just 20% of a transformer's middle layers—rather than all layers—can improve reasoning ability with minimal inference cost, and this works even when retrofitted onto already-trained models.

This paper introduces T²MLR, a technique that adds recurrent connections to transformer models by caching middle-layer representations and feeding them back into earlier layers during decoding.

Weishuai Zeng, Kangning Yin, Xiaojie Niu et al.

Scaling humanoid robot control effectively requires coordinating learning paradigm, data diversity, and model architecture—not just increasing scale. The motion-tracking formulation and Humanoid Transformer enable structured behavioral learning that generalizes across diverse tasks.

This paper presents a scaling recipe for Behavior Foundation Models applied to humanoid robot control. The key innovation is coordinating three components: a motion-tracking learning paradigm that treats control as reproducing whole-body behaviors, strategic balance between on-policy data collection and motion diversity, and a new Humanoid Transformer architecture.

Jiajun Hu, Ruthwik Reddy Sunketa, Lei Zhao et al.

Replacing ADCs with analog content-addressable memories in FPGA-integrated IMC blocks enables efficient inference on Transformer models by natively supporting nonlinear operations and dynamic matrix multiplication, not just static-weight computation.

This paper proposes NIFA, an FPGA architecture that integrates analog in-memory computing (IMC) blocks with analog content-addressable memories (ACAMs) to accelerate deep learning inference.

Sridhar Mahadevan

Machine learning can be understood as searching for stable points where infinitesimal changes preserve compositional structure—a perspective that unifies deep learning, LLMs, and RL under a single categorical framework.

This paper proposes LINCS, a categorical framework that reformulates machine learning as a problem of fixing compositional failures in abstract mathematical structures called sketches.

William Mocaër, Solène Tarride, Thomas Constum et al.

Tiramisu demonstrates that explicitly modeling document hierarchy through tiered transformer attention can outperform traditional bottom-up pipelines for understanding complex newspaper layouts, offering a scalable alternative for historical document digitization.

This paper tackles understanding complex newspaper layouts by proposing two approaches: a modular pipeline combining existing models (YOLO, LayoutReader) for layout detection and reading order, and Tiramisu, a novel transformer architecture that hierarchically processes newspaper structure end-to-end.

Zhan Chen, Jiqiao Ma, Chih-wen Kuo

You can build effective multi-domain systems for low-resource languages by reusing intermediate checkpoints as specialists and routing based on visual style, rather than training separate models from scratch for each domain.

This paper tackles historical Manchu text recognition across multiple visual styles (regular script, running script, palace memorials) with limited training data. The authors build a multi-expert system that reuses fine-tuned model checkpoints as domain specialists and uses a lightweight image classifier to route pages to the appropriate expert.

Katie Everett

Architecture design for deep networks is fundamentally about balancing rank preservation (keeping information flowing), ensemble-like behavior (layers staying independent), and parameter efficiency—skip connections and normalization placement control this tradeoff by managing how much gradient ...

This paper reveals how Transformer architecture components preserve information flow across deep networks by maintaining mathematical rank. Skip connections and layer normalization prevent rank collapse—where information gets squeezed into fewer dimensions—by routing gradients around lossy operations.

Jorge Diaz Chao, Konpat Preechakul, Yuxi Liu et al.

Video diffusion models lack scalable serial computation for sequential reasoning tasks—denoising steps don't add computational depth beyond the backbone network, making them structurally limited for physics simulation and causal prediction.

Video diffusion models struggle with tasks requiring sequential reasoning, like predicting multi-ball collisions. Researchers show this 'seriality gap' stems from the models' inability to scale serial computation as causal chains grow longer, even with more denoising steps. Autoregressive and deeper architectures help, but standard bidirectional diffusion hits a fundamental limit.

Hongru Cai, Yongqi Li, Ran Wei et al.

Running agents natively on mobile devices with direct access to device capabilities (via structured APIs) is 11.5% more successful and 95% faster than GUI-based automation approaches.

PalmClaw is a framework that runs AI agents directly on mobile phones, letting them control device features like contacts, calendar, and sensors through structured tool calls instead of simulating screen taps. This approach is faster and more reliable than existing mobile agents that interact through GUI actions.

Harsha Vardhan Khurdula, Abhinav Kumar Singh, Yoeven D Khemlani et al.

Parallel decoding via discrete diffusion is viable for speech recognition and can match autoregressive performance while being faster and more efficient, using only 0.16% trainable parameters on a frozen backbone.

This paper shows that discrete diffusion language models can transcribe speech by refining entire transcripts in parallel rather than generating one token at a time. The authors adapt DiffusionGemma (a 26B model) for speech by freezing most weights and adding a lightweight audio interface, achieving competitive accuracy while transcribing in just eight parallel steps regardless of speech length.

Junrui Zhang, Zemin Chen, Lusi Li et al.

Quantum Neural Networks can be attacked with input-specific backdoors that are harder to detect than fixed-trigger attacks, and current QNN defenses are insufficient against this threat.

This paper introduces Q-DIBA, the first input-aware dynamic backdoor attack against Quantum Neural Networks. Unlike previous quantum backdoor attacks that use fixed triggers, Q-DIBA generates unique triggers for each input by jointly training a classical trigger generator with the victim QNN.

Romain Amigon

You can now search for efficient neural architectures on a standard consumer GPU by combining RL-trained Transformers with swarm optimization, discovering models 10-100x smaller than standard baselines while maintaining competitive accuracy.

This paper presents a hybrid Neural Architecture Search (NAS) framework combining a Transformer-based reinforcement learning controller with an Artificial Bee Colony algorithm to design efficient deep learning models on consumer GPUs.

Emanuele Quinto, Carlo Andrea Rozzi, Francesco Zanitti

Workflows can be represented as first-class knowledge objects that persist and remain queryable, making it easier to inspect, resume, and audit LLM-based processes—moving beyond treating workflows as black boxes that just produce outputs.

This paper proposes a conceptual model for LLM workflows that treats workflow definitions, instances, and execution traces as persistent knowledge objects. It distinguishes between deterministic computation (derive) and LLM-mediated judgment (infer), enabling workflows to be inspectable, resumable, and reviewable rather than just producing outputs and leaving traces.

Ayda Eghbalian, Kevin Desai

You can now add biomechanical analysis (forces, activations, loads) to any existing 3D pose estimator without retraining it, making markerless motion capture useful for rehabilitation, sports, and clinical applications.

This paper introduces BioModule, a lightweight transformer that converts 3D skeletal poses from any pose estimator into biomechanical quantities like joint forces and muscle activation. By aligning Human3.6M video data with biomechanical labels, the authors show how pose estimation errors propagate to biomechanical predictions across seven different pose estimators.

Anna Kuzina, Paul N. Whatmough, Babak Ehteshami Bejnordi

Linear attention mechanisms can match standard transformer performance if you design the state updates correctly; the paper shows which architectural choices matter most for maintaining accuracy while cutting inference cost.

This paper analyzes why transformer self-attention is expensive and proposes a linearized alternative that reduces computational cost from quadratic to linear. By studying how attention mechanisms work mathematically, the authors identify key design principles—like using delta-style updates and sink tokens—that preserve model quality while dramatically speeding up inference on long documents.

Xiangming Huang, Guannan Zhang, Lu Lu et al.

Neural operators can compress high-dimensional design spaces into low-dimensional latent representations that preserve physics-aware structure, making evolutionary optimization practical for inverse design problems that would otherwise be intractable.

This paper combines neural operators with evolutionary optimization to solve inverse design problems for physical systems governed by PDEs. By learning a compact representation of design space topology and coupling it with CMA-ES, the method reduces design dimensionality dramatically while maintaining high performance across different operating conditions.

Eitan Levin, Venkat Chandrasekaran

Models trained on small inputs can generalize to larger ones if they're continuous with respect to appropriate sampling operations—the paper provides explicit rates and identifies which sampling strategies (replacement, binning, species sampling) work for different problem types.

This paper develops a unified framework for understanding how machine learning models generalize from small to large inputs of variable sizes (like point clouds or graphs). Using random sampling maps, the authors characterize when models can reliably extrapolate to unseen input sizes and how to compress large inputs while preserving model predictions.

Tianjiao Yu, Xinzhuo Li, Yifan Shen et al.

By routing text-to-geometry interactions across matched abstraction scales rather than flattening everything into one representation, you can build faster, more accurate 3D models that understand both coarse structure and fine details.

ELSA3D is a unified 3D foundation model that improves how language and 3D geometry interact by using 'anchor tokens' to match text concepts with the right level of geometric detail. Instead of treating all information equally, it routes language features to specific scales of 3D structure, making the model both more efficient and more accurate for generating and understanding 3D objects.

Shervin Khalafi, Igor Krawczuk, Sergio Rozada et al.

Linear attention in graph transformers can only learn averaged denoising filters, but Graph Convolutional Attention leverages spectral information to adapt denoising to each graph's unique structure—improving both performance and inference speed.

This paper explains why standard attention mechanisms struggle with graph denoising and proposes Graph Convolutional Attention (GCA), which uses the graph's spectral properties to denoise more effectively. GCA provably outperforms linear attention and works well in graph diffusion models, offering both theoretical guarantees and practical speedups.

Zhenyu Liu, Yunxin Li, Xuanyu Zhang et al.

Modality interference—caused by gradient conflicts between audio and semantic processing—is the root cause of poor full-duplex SLM performance; hierarchical parameter separation solves this while maintaining cross-modality coherence.

This paper identifies and solves a critical problem in full-duplex spoken language models: when audio and text processing share the same neural network layers, they create conflicting gradients that degrade performance.

Ghassen Marrakchi, Basarab Matei

By separating how patient data flows through a system from how AI models process it, you can build more maintainable clinical AI that adapts to real hospital environments without rewriting core logic.

This paper presents the Large Cancer Assistant (LCA), a flexible orchestration framework that decouples data handling from AI models in cancer diagnosis systems.

Uwe M. Borghoff, Paolo Bottoni, Remo Pareschi

Hardware-enforced coordination can provide hard safety guarantees for autonomous systems by implementing coordination rules at the FPGA level, separating deterministic interaction management from adaptive AI reasoning.

This paper proposes using FPGAs to enforce safety-critical coordination rules directly in hardware for autonomous systems that combine AI models with real-time control. By moving coordination logic from software to hardware, the system guarantees deterministic timing and verifiable safety constraints while keeping AI reasoning flexible.

Giovanni Monea, Nathan Godey, Kianté Brantley et al.

Splitting Transformer computation into separate streams for token prediction and state maintenance improves both training efficiency and model performance—a simple architectural change with consistent gains across scales.

This paper proposes separating two functions in Transformers: predicting the next token and maintaining state for future predictions. The authors design a dual-stream architecture and show it improves language modeling efficiency and downstream task performance by 2-3% compared to standard Transformers.

Patrick Podest, Marco Pichler, Elias Bürger et al.

TiRex-2 enables efficient streaming multivariate forecasting with constant per-patch inference cost and zero-shot generalization, solving the quadratic complexity problem of Transformer-based time series models.

TiRex-2 is a time series foundation model built on xLSTM that handles multiple variables and streaming data efficiently. Unlike Transformer-based models that slow down with longer sequences, TiRex-2 uses a memory-based recurrent design that processes new data at constant cost, even when variables evolve together and some future values are known in advance.

Jian Gu, Aldeida Aleti, Chunyang Chen et al.

SemRF provides a principled way to track semantic meaning across model layers, enabling researchers to distinguish real computation from measurement drift and connect layer-wise complexity to parameter efficiency.

This paper introduces Semantic Reference Frames (SemRF), a mathematical framework for analyzing how language models process information across layers.

Shun Lei, Huaicheng Zhang, Dapeng Wu et al.

For music generation at scale, separating semantic planning (what to generate) from acoustic refinement (how to generate it) and training them sequentially rather than simultaneously improves both coherence and sound quality.

LeVo 2 generates full-length songs by combining language models and diffusion models in a hierarchical approach: first predicting mixed vocal-instrument tokens for overall coherence, then refining each track separately for acoustic detail.

Liyao Wang, Ruipu Wu, Haojun Xu et al.

Combining explicit 3D geometry (camera poses, spatial relationships) with visual matching dramatically improves cross-view localization and enables zero-shot transfer between ground and drone views without paired training data.

This paper tackles cross-view object geo-localization—finding a target object in satellite imagery when given a ground or drone photo. The authors introduce a large dataset with 220K+ image pairs and geometric metadata, plus GAGeo, a unified framework that predicts object locations, masks, and camera poses simultaneously using 3D spatial understanding rather than just appearance matching.

Wei Zhou, Xiongwei Zhu, Zelin Xu et al.

Multi-task image generation models can be trained more effectively by treating each capability (T2I, local edit, global edit) as a separate velocity field and having the student learn to compose them on its own generated trajectories.

DanceOPD is a training framework that helps image generation models master multiple tasks—text-to-image, local editing, and global editing—without them interfering with each other. It uses a distillation approach where a student model learns from specialized 'capability fields' (velocity fields in flow-matching models), routing each image to the right expert for its task.

Danyal Rehman, Charlie B. Tan, Yoshua Bengio et al.

Autoregressive models can outperform flow-based approaches for molecular sampling by avoiding invertibility constraints and enabling better scalability—opening a new direction for physics-informed generative modeling.

This paper introduces Autoregressive Boltzmann Generators (ArBG), a new method for efficiently sampling molecular systems at equilibrium. Unlike previous approaches using normalizing flows, ArBG uses autoregressive models to generate molecular configurations faster and more accurately, with a large pre-trained model (Robin) achieving 60% better energy predictions on peptide systems.

Yu-Neng Wang, Sara Achour

Analog hardware can generate images 100x more efficiently than digital systems, but requires rethinking model design to match fixed physics-based dynamics rather than flexible neural networks.

This paper proposes Analog Interaction Systems (AIS), a framework for building generative models on analog hardware like coupled oscillators. The key innovation is bridging the gap between what neural networks can do and what analog physics naturally computes—using time-varying parameters and hidden states to improve expressivity while keeping energy costs 100x lower than digital approaches.

Andrew Bond, Zafer Doğan

Structured correlations in data can boost learning of weak features in GANs, but excessive correlation destabilizes training—there's a sweet spot determined by learning rates and noise.

This paper analyzes GAN training mathematically by studying how a linear generator learns data structure. The key innovation is handling realistic data with correlated features and class-dependent patterns—not just simple diagonal structure. The authors prove training converges to predictable equations and show that smart use of correlations can help weak features become learnable.

Sayak Dutta

Recurrent models can match Transformer efficiency by making forget gates content-aware (looking at stored memory) rather than memory-blind, enabling a mathematical solver that speeds up training while improving language understanding.

CARVE improves recurrent neural networks by fixing how they decide what to forget. Instead of gates that only see new incoming data, CARVE's gates look at what's already stored in memory before deciding what to erase. This single change fixes three architectural problems, enables faster training, and achieves better performance on language tasks while using less memory than competing approaches.

Ping Xiong, Thomas Schnake, Klaus-Robert Müller et al.

When explaining temporal graph models, you need to track information flowing through event-induced variables—not just embeddings—to capture how long-range dependencies actually work in the network.

This paper develops a new method to explain how Temporal Graph Neural Networks make predictions by tracking information flow through all components, not just embeddings. The approach uses a framework called Normalized Relevance Measure to systematically decompose complex temporal graph models and identify which events and interactions matter most for predictions.

Fabiana Fournier, Lior Limonad

You can add agentic AI capabilities to existing business processes by wrapping them with policy-governed agents at specific control points, rather than rebuilding the entire system.

This paper introduces a 'process harness'—a layer that adds AI reasoning to existing business workflows without replacing them. It uses three types of AI agents (for tasks, decisions, and flow control) that operate within policy guardrails, letting legacy systems maintain structural control while gaining adaptive intelligence at key decision points.

Haorui Ji, Weizhe Liu, Hongdong Li et al.

To build better image-to-3D systems, focus on using features designed for reconstruction rather than classification, and explicitly align 2D image information with 3D geometry during generation.

FLUX3D generates high-quality 3D Gaussian Splatting models from images by improving how 2D image features are converted into 3D representations and how the generation process aligns 2D and 3D data. It uses specialized techniques to preserve fine visual details that previous methods lost, resulting in better-looking 3D assets.

Jason Sulskis, Sathya Ravi

For PDE solvers, choose your spectral basis based on the operator's symmetry: real bases for elliptic PDEs, complex bases for time-dependent ones with phase content.

This paper compares two neural operators for solving PDEs: Fourier Neural Operators (FNO) using complex FFT and Hartley Neural Operators (HNO) using real Hartley transforms. Both are iso-parametric but use different spectral bases.

Zixuan Li, Haokun Lin, Yicheng Xiao et al.

Separating structure planning from appearance rendering in image generation improves prompt following for complex spatial and compositional requirements without needing intermediate outputs.

This paper improves text-to-image generation by separating structural planning from appearance rendering. IV-CoT uses two types of queries—structural and semantic—that work together in a single pass: structural queries create a latent visual plan (like an invisible sketch), then semantic queries render the final image based on that plan.

Zidu Liu, Florian Marquardt

LLMs can solve discrete design problems in quantum computing by evolving structured concepts rather than generating solutions from first principles—showing that domain-specific constraints and executable specifications make AI search more effective.

Researchers used large language models paired with structured algebraic rules to automatically discover new quantum error-correcting codes. Instead of designing codes from scratch, the system evolves mathematical specifications and programs that describe code families, finding competitive designs that work better than some existing approaches.

Reza Bayat, Ali Behrouz, Aaron Courville

You can improve language model efficiency by tapering MLP width across depth—allocating more capacity to early layers and less to later ones—a free performance gain that works across different architectures.

This paper shows that language models waste parameters by allocating them uniformly across layers. The authors propose Tapered Language Models, which gradually reduce the width of MLPs (the largest parameter-consuming components) from early to later layers. Across multiple architectures and scales, this simple change improves performance without extra cost.