Papers

Cheng Xu, Nan Yan, Liming Chen et al.

When evaluating whether models improve on individual problems, you need a separately measured null baseline for every statistic—not just comparing two noisy estimates—or you'll mistake measurement artifacts for real capability gains.

This paper audits claims about language model self-improvement by comparing a fine-tuned model against a frozen control run through identical evaluation pipelines. The authors identify seven measurement artifacts that flip reported findings, showing that many apparent capability gains are statistical illusions from batching effects and noisy comparisons rather than real improvements.

Qian Kou, Xiaofeng Shi, Xiaosong Qiu et al.

To internalize document knowledge without retrieval, separate the training into three stages: inject documents as training data, align with QA supervision, then merge back with the base model to avoid catastrophic forgetting of general skills.

This paper tackles the problem of making LLMs answer questions about specific documents without retrieving them at inference time. The authors propose IAR, a three-stage training method that injects document knowledge, aligns the model to answer questions correctly, and recovers general abilities by merging with the base model.

Yiyang Feng, Biddut Sarker Bijoy, Niranjan Balasubramanian et al.

Subtask-level skills expressed as text transfer more reliably across tasks than task-level or code-based skills, and you can predict skill usefulness by balancing how specific it is versus how broadly applicable it is.

This paper studies how LLM agents can learn and reuse skills from completed tasks. The researchers compare different ways of capturing skills—at the task level versus subtask level, and as text versus code—and find that breaking skills into subtasks and expressing them as text leads to better transfer across new tasks.

Bo Liu, Simon Yu, Yiding Jiang et al.

Making environment design itself learnable—rather than hand-curated or static—enables continuous self-improvement in language agents by automatically generating appropriately-difficult, diverse training tasks.

SPADE is a self-play framework where a single language model learns two roles: designing custom training environments as executable code, and solving problems within them. The environment designer learns to create challenges at the edge of the agent's abilities, automatically adapting as the agent improves.

Jayjun Lee, Jessica Yin, Asif Rana et al.

Pre-training on a generic manipulation task and using a stable fine-tuning recipe with behavior cloning and conservative updates enables efficient transfer of dexterous skills to new tasks and real robots without catastrophic forgetting.

ADEPT is a reinforcement learning framework that trains dexterous robot hands to perform complex manipulation tasks by first learning a general object-handling skill, then adapting it to specific downstream tasks. The system transfers from simulation to real robots with multi-fingered hands (23-29 degrees of freedom) using vision and touch sensors, solving long-horizon tasks at human-level speed.

Zhu Zhang, Jixun Wang, Xiaoang Xu et al.

When training smaller models on long-context tasks, combining token-level teacher guidance with task-level verifier rewards—by measuring and distributing their disagreement—works better than either signal alone.

This paper addresses a key problem in training smaller language models for long-context tasks: teacher models give token-level guidance that looks locally correct but misses global evidence or violates task constraints.

Aditya Bhattacharjee, Christos Plachouras, Sungkyun Chang et al.

Combining contrastive and triplet losses with semi-hard negatives on a lightweight encoder (MobileNetV3) effectively learns to match vocal imitations to sound effects, balancing accuracy with deployment efficiency.

This paper describes winning approaches for a challenge on finding sound effects by imitating them with your voice. The team tested two methods: one using a frozen pre-trained audio encoder with contrastive learning, and another using a mobile-friendly encoder trained with both contrastive and triplet losses. Both approaches help match vocal imitations to actual sound recordings.

Zachary Speck, Asa Shepard

Individual training examples can be learned and then completely forgotten during pre-training, leaving virtually no measurable impact on final model behavior or internal structure, suggesting that what matters for final performance is the aggregate signal, not individual data points.

Researchers trained 32 GPT-2 models from scratch and injected a single training example at peak learning rate to measure its actual impact. The example was learned immediately but completely forgotten by the end of training, leaving no detectable trace in the final model's weights, geometry, or performance—despite moving the model within its loss basin during training.

Tomasz R. Bielecki, Thibaut Mastrolia, Haoze Yan

You can apply continuous-time reinforcement learning to complex, memory-dependent systems by first converting them to Markovian approximations—this opens up RL applications to event-driven processes like financial markets and network dynamics.

This paper tackles reinforcement learning for systems driven by Hawkes processes—mathematical models where events cluster in time and influence future events. The key challenge is that these systems have memory and aren't Markovian, so standard RL doesn't apply.

Jiaming Li

When you have a fast but imperfect analytical model and limited expensive simulation data, teaching a neural network to correct the analytical model's errors—or pre-training it on the analytical model first—can cut your data requirements dramatically.

This paper shows how to make machine learning models more data-efficient by combining cheap analytical equations with expensive high-fidelity simulations. Using Helmholtz resonators as a test case, the authors demonstrate two approaches: learning to correct analytical predictions, or distilling analytical knowledge into a neural network before fine-tuning with limited simulation data.

Minh-Ha Nguyen, Cathy Shyr

You can improve a frozen language model's performance on specialized tasks by treating policy refinement as a human-in-the-loop process: have an AI critic identify recurring failures, propose natural-language policy changes, and let domain experts decide what gets deployed.

This paper presents Policy Iteration with Human Feedback (PIHF), a method that improves a fixed language model's performance on rare-disease diagnosis by iteratively refining its decision-making policy through human expert review.

Ben Anson, Conor Houghton, Edward Milsom

Muon can now be used effectively with LoRA-style low-rank fine-tuning by approximating its solution mathematically, offering a practical way to combine two previously incompatible techniques.

This paper solves a mathematical incompatibility between Muon (an optimizer that works well for pretraining) and LoRA (a popular parameter-efficient fine-tuning method). The authors propose sMuon, which approximates Muon's benefits in the low-rank setting through linearization and least-squares, using only standard matrix operations.

Alexy Skoutnev, Kirill Acharya, Gaston Longhitano et al.

Test-time world model inference—where an AI builds and validates its own simulation of unknown environments—dramatically improves task performance (from 7.8% to 93.3%) and shows that learning game rules is easier than inferring goals.

Twin is a system where an AI agent builds executable world models to solve puzzle games by observing and simulating gameplay. Rather than hand-coding rules for each game, the system learns the game's mechanics through trial and error, validates predictions against actual outcomes, and fixes mismatches. It solves 97.8% of test games more efficiently than humans.

Abhishek Shukla, Ankur Sinha, Faiz Hamid

By formulating architecture search as a linear program using gradient and curvature information, LP-NAS finds better neural network designs 2-3x faster than standard differentiable NAS methods while achieving higher accuracy.

This paper proposes LP-NAS, a neural architecture search method that uses linear programming to find better network designs faster. Instead of randomly exploring architectures, LP-NAS uses mathematical optimization principles to guide the search, resulting in architectures that generalize better and are found more quickly than existing methods like DARTS.

Fanfei Li, Jana Zeller, Manuel Prada-Corral et al.

Training on a carefully curated, grade-level-appropriate curriculum creates a sandbox for studying knowledge acquisition with clear boundaries—useful for understanding how models learn and what happens when you try to teach them new concepts.

Researchers created LittleLeaner, a 5B-parameter language model trained on an 88B-token curriculum limited to U.S. Grade 5 material, to study how models acquire knowledge under controlled conditions. Unlike models trained on messy web data, LittleLeaner has clear, interpretable knowledge boundaries, making it easier to understand what the model knows and how it learns new information.

Weihan Meng, Hongzhu Guo, Yi Jing et al.

You can teach an LLM to explain what individual features in another LLM do by injecting those features and training it to verbalize their meaning—this works across different SAE dictionaries and even different models.

SAEVerbalizer trains language models to generate natural-language explanations for features extracted by sparse autoencoders (SAEs). Instead of observing model behavior to understand what features do, the system directly injects feature directions into the model and fine-tunes it to produce explanations, making the process faster and more scalable.

Tianyi Li, Yaxin Luo, Xinyi Shang et al.

DARTree achieves up to 9.73× speedup on language model inference by constructing and scoring candidate token trees in parallel, accepting nearly 13 tokens per verification step—nearly 3× more than prior methods.

DARTree speeds up language model inference by using a tree-based speculative decoding approach. Instead of generating one draft token at a time, it predicts multiple token candidates in parallel using diffusion, then verifies them together. The method uses an autoregressive correction head to score entire branches at once, accepting more tokens per verification round than existing methods.

Martin J. Wainwright

UGC provides a principled way to design adaptive masking schedules for discrete diffusion that are provably near-optimal, with potential √d speedups over fixed schedules by concentrating computational effort where data geometry demands it.

This paper introduces unmasking growth complexity (UGC), a geometric measure that controls how discrete diffusion models should reveal information during sampling. The key insight is that UGC increments directly bound discretization error, enabling optimized sampling schedules that adapt to data structure.

Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina et al.

You can build competitive frontier-level language models at 1B parameters using only openly licensed data, making it feasible for researchers and organizations to develop ethical AI without relying on scraped or restricted datasets.

Mimir v1 is a 1-billion-parameter language model trained entirely on permissible (legally and ethically sourced) data that achieves competitive performance with much larger models. It uses a Hierarchical Reasoning Model architecture and excels at English, math, code, and Danish tasks—showing that high-quality open-source models don't require massive proprietary datasets.

Yuto Nishida, Hirokazu Kiyomaru, Yusuke Oda et al.

You can measure training data importance during pretraining by tracking parameter trajectories, revealing that different data types matter at different training stages—literature early, STEM later—without needing task-specific validation sets.

This paper introduces a new method to measure how much training data influences language model development without needing to pick specific downstream tasks. Instead of testing on particular benchmarks, the researchers measure influence by tracking how each piece of training data pushes the model toward its final parameters.

Yi-Chung Chen, Philip Jacobson, Tom Lampo et al.

Using physical trajectory data as training supervision helps video embedding models better understand motion-centric driving events, improving retrieval accuracy by 5-10% while keeping inference simple and efficient.

This paper tackles retrieving relevant driving video clips from large datasets by improving multimodal embedding models. The key innovation is TraVEL, which fine-tunes video embeddings using trajectory (vehicle motion) as training supervision, helping the model understand motion-centric events like turning or accelerating.

Julian Minder, Viktor Moskvoretskii, Raghav Singhal et al.

Installing alignment values during pretraining from the beginning creates deeper, more robust alignment than adding it after training, and this advantage grows with more pretraining data.

This paper introduces Synthetic Persona Pretraining (SPP), a method that embeds desired values and assistant behavior directly into language models from the start of pretraining rather than adding them afterward.

Yuchen Xin, Zhihua Zhang

For sampling from nonsmooth targets, tracking local curvature via the active trace gives tighter convergence guarantees than worst-case dimension-dependent bounds, achieving O(ε⁻²) instead of O(ε⁻³) for common structured problems.

This paper analyzes the Moreau-Yosida unadjusted Langevin algorithm (MYULA) for sampling from nonsmooth distributions. The key contribution is showing that the algorithm's convergence depends on an 'active trace' quantity (measuring local curvature) rather than global dimension, leading to improved complexity bounds—especially for structured penalties like lasso and total variation.

Irina Proskurina, Mayank Kumar, Oyindolapo O. Komolafe

Instruction tuning makes models sound more confident without improving accuracy, and it reduces the diversity of explanations they provide—a potential concern for transparency and reliability.

This paper investigates how instruction tuning affects language models' confidence levels and the diversity of their explanations. The researchers found that instruction tuning makes models express higher confidence in their answers, but this doesn't match improvements in actual accuracy.

Shivam Vats, Sudarshan Harithas, Mete Tuluhan Akbulut et al.

When robots have limited time to practice, you can use optimization to compute exactly which skills to learn and how long to spend on each—trading off learning difficulty against the reward value each skill unlocks.

This paper tackles how robots can learn skills efficiently when practice time is limited. The authors propose Deliberate Practice, an algorithm that decides which skills to practice and for how long to maximize task performance within a fixed budget. They use a mathematical optimization approach to find the best allocation of practice time across different skills.

Cheng Qian, Wenting Zhao, Liangwei Yang et al.

You can transfer capabilities from strong to weak models at inference time through clever prompt engineering and code scaffolding, achieving similar gains to training-time distillation without any parameter updates.

This paper shows that stronger AI models can help weaker ones solve tasks better at test time by building 'harnesses'—structured inference-time scaffolds like code templates and routing logic—without retraining. Testing on reasoning benchmarks, the approach nearly doubles weaker model performance by offloading unstable reasoning into deterministic code and enforcing strict output formats.

Shiyu Xuan, Zechao Li

GUI agents can now self-improve after deployment by learning from their own failures through AI-guided reflection and self-distillation, achieving 7.4% accuracy gains without requiring human annotations.

This paper introduces a test-time adaptation framework for GUI visual grounding that allows models to improve after deployment without human feedback. The system uses a closed-loop process: agents explore interfaces, an AI reflector evaluates results and explains failures, and a self-distillation method internalizes these insights back into the model weights.

Changhao Xiang, Shangyu Xing, Zhen Wu et al.

By interleaving visual objects directly into text during pretraining, you can teach multimodal models object-level grounding 12x more efficiently than traditional image-text pair training.

This paper introduces MultiModal Code-Switching (MMCS), a new way to train vision-language models by replacing words in text with their corresponding visual objects. Instead of just pairing whole images with descriptions, MMCS explicitly shows the model which objects match which words, making training much more efficient—achieving the same performance with 12x less data.

Nguyen Thai Anh, Truong Viet Vu, Tran Thien Thanh et al.

Adaptive smoothing in Naive Bayes can reduce prediction errors by up to 22% on high-cardinality data and improve calibration by 41-70% compared to standard fixed-smoothing approaches, with theoretical guarantees that match optimal rates.

This paper improves Naive Bayes classifiers by replacing fixed smoothing rules with a data-adaptive approach that learns smoothing strength from the data itself. The method uses hierarchical empirical Bayes to adjust how much prior information to add based on feature cardinality and sample size, achieving better probability estimates and calibration on real-world datasets.

Zetao Hong, Song Yuan, Yuanhao Ding et al.

When running multiple RL training pipelines on shared inference hardware, intelligent scheduling of which requests to accept and how to allocate GPU memory between them can dramatically improve efficiency without compromising training quality.

This paper addresses a scheduling problem in LLM inference services that run multiple types of reinforcement learning workloads simultaneously. Different RL approaches (RLVR, RLHF, agentic) have different memory and timing needs, causing them to compete inefficiently for GPU cache.

Congfeng Cao, Pengyu Zhang, Jelke Bloem

Training models for both fast responses and deep reasoning requires careful balancing—prioritizing one mode degrades the other, and the optimal training schedule depends on how much data you allocate to each mode.

This paper studies how to train language models that can switch between quick answers and detailed reasoning. The researchers found that training these two modes together creates a trade-off: focusing more on quick answers hurts the model's ability to reason deeply. They tested different training schedules and data ratios to find the best balance for math problems.

Björn Engdahl, Adrian Kosowski, Jan Chorowski et al.

Recurrent latent reasoning (thinking without verbalizing) combined with in-context learning enables smaller models to solve complex reasoning tasks more cost-efficiently than larger models that explain their work.

BDH-CQ is a reasoning model that learns from examples at inference time and solves problems through hidden, iterative computation rather than step-by-step explanations. On the ARC-AGI benchmark, a small 150M-parameter version achieves 29.5% accuracy while costing just $0.0007 per task—better cost-efficiency than previous approaches.

Wanying Qu, Qinghua Mao, Yu Li et al.

Instead of treating agent safety controls as static, you can automatically improve them by analyzing what goes wrong in practice and evolving specific components like rules and tool permissions—achieving 3x better safety without sacrificing utility.

This paper introduces SHE, a framework that automatically improves LLM agent safety by evolving the agent's control system (harness) rather than just the model weights. It breaks the harness into four modular components—system prompt, rules, memory, and tool access—and learns safer versions of each from failure examples, while maintaining the agent's usefulness.

Xu Ouyang, Moontae Lee

Language models can be trained to output financial predictions and portfolio weights as constrained tokens, matching or beating traditional regression-based approaches while maintaining a unified model architecture.

This paper shows that language models can directly generate financial forecasts and investment decisions as tokens, rather than using separate prediction heads. FinATOM uses constrained token generation to predict stock returns and allocate across ETFs, achieving better risk-adjusted returns than traditional approaches when trained with ranking supervision and reinforcement learning.

Mingxuan Zheng, Yujin Zhou, Chuxue Cao et al.

Agents can evolve skills more effectively by coupling failure diagnosis with proximal optimization: diagnose what went wrong, test fixes on the same tasks, then systematically audit and remove unhelpful skill components.

SkillProx improves how AI agents learn and refine reusable task skills by combining diagnostic feedback loops with a mathematical optimization approach. Instead of treating skill edits generically, it explicitly diagnoses failures, rolls back unsuccessful changes, breaks skills into auditable components, and removes or demotes unhelpful knowledge—improving task accuracy by 3 percentage points.

Ali Janati, Kaoutar El Maghraoui, Andrei Kanavalau et al.

Even when an optimizer finds a correct solution to a task, different optimizers can cause the solution to collapse post-training due to mismatched step-size dynamics at the representation-readout interface—a failure mode invisible to standard loss curves.

This paper investigates why Muon-optimized transformers solve modular arithmetic tasks during training but then catastrophically lose generalization afterward. The authors identify the failure occurs at the interface between learned representations and the output layer, where different optimizer dynamics cause instability.

Zhaoyu Zhu, Rui Gao, Shuang Li

Wasserstein policy gradient converges exponentially fast for entropy-regularized linear-quadratic control without exponential slowdown as regularization decreases, making it a theoretically sound alternative to standard policy gradient methods.

This paper studies how to optimize control policies using Wasserstein gradients—a method that updates action distributions by moving them in action space. For linear-quadratic control problems with entropy regularization, the authors prove that this approach reduces to a simple finite-dimensional system that converges reliably to the optimal policy, even as the regularization strength changes.

Xian Sun, Wei Chow, Yingshuo Wang et al.

The goal isn't to make models resistant to all external signals, but to teach them when to trust context and when to ignore it—a nuanced skill that requires balanced training across clean, misleading, and irrelevant contexts.

Language models often blindly follow external context, even when it's wrong. This paper introduces MIST, a benchmark that tests when models should trust context versus ignore it, and SCOPE, a training method that teaches models to selectively trust helpful context while rejecting misleading signals—without becoming useless when context is actually correct.

Fanzhe Meng, Guoxin Chen, Jiale Zhao et al.

Training agents on tasks calibrated to be appropriately difficult (not too easy, not impossible) using multiple solver feedback produces better generalization than manually authored or single-solver validated tasks.

CalibForge automatically creates training tasks for AI agents by using multiple solvers to identify tasks that are challenging but solvable—the 'learnable zone.' It revises candidate tasks based on solver disagreement and performance patterns, then trains agents on these calibrated tasks, achieving significant improvements on code and repository understanding benchmarks.

Xinye Wang, Junxiao Liu, Shujian Huang

By identifying and prioritizing training on critical reasoning decisions (pivots) rather than all tokens equally, multilingual reasoning transfer becomes more efficient and effective across 17 languages.

This paper improves how large language models learn to reason in multiple languages by focusing training on the most important decision points in reasoning—called 'reasoning pivots'—rather than treating all tokens equally.

Daniel Paulin, Victor Elvira

VARMA models are now computationally practical for high-dimensional time series because optimization cost no longer scales with series length—enabling better forecasts than VAR models on real data.

This paper solves a long-standing computational problem in time series modeling: estimating VARMA models (which capture dependencies efficiently) at scale. The key innovation is making each optimization step independent of series length by using Fourier-based sufficient statistics and a reparametrization that ensures valid models by construction.

Chenglong Wang, Ziming Zhu, Yifu Huo et al.

Generative reward models work better in RL when you extract rewards from their ranking capabilities rather than forcing them to produce scalar scores—use pairwise or reference-based comparisons instead.

This paper shows how to use generative reward models (which rank responses) effectively in reinforcement learning for language models. The key insight is that generative models naturally compare responses rather than score them individually.

Yijiang Li, Bingyang Wang, Yijun Liang et al.

You can improve LLMs through self-distillation using only the model's own outputs and internal consistency, without needing ground-truth labels or external feedback—making it truly self-supervised post-training.

This paper introduces Unsupervised On-Policy Self-Distillation (U-OPSD), a method that improves language models without requiring external supervision like ground-truth answers or feedback from larger models.

Johann Birnick, Rayan Saab

You can now quantize neural networks with better geometric information (two-sided Hessian) at roughly the same computational cost as simpler one-sided methods, enabling more accurate model compression.

BaKron speeds up neural network quantization by efficiently using two-sided Hessian information to round weights more intelligently. Instead of the expensive O(m²n²) computation, it achieves O(mn(m+n)) work while maintaining the same cubic scaling as existing methods, making it practical to apply richer curvature information during quantization.

Iosif Lytras, Nikolaos Makras, Sotirios Sabanis

SG-TULA provides a theoretically-grounded alternative to standard optimizers for non-smooth, non-convex problems with explicit convergence guarantees—useful for understanding and improving LLM pretraining when standard assumptions break down.

This paper introduces SG-TULA, a sampling algorithm for training machine learning models when the loss landscape is non-smooth, non-convex, and has steep gradients. Unlike existing methods, it works directly with subgradients without expensive smoothing, uses taming techniques for stability, and comes with theoretical guarantees on convergence speed.

Robin Trombetta, Carole Lartizien

Using optimal transport to blend medical images creates more realistic and varied synthetic lesions than traditional mixing strategies, leading to better segmentation model performance.

This paper presents OTLesMix, a data augmentation method that uses optimal transport and Wasserstein barycenters to generate synthetic medical images with diverse lesion shapes and locations. Tested on brain lesion segmentation, it improves model performance by 2.9-6.6 Dice points compared to baseline and outperforms existing mixing-based augmentation methods.

Yiting Zheng, Cheng Fang, Anthony Donofrio et al.

Using a unified graph representation of reactions instead of separate reactant/product graphs enables better learning of chemical transformations, leading to more accurate yield predictions and a foundation model that could generalize across diverse reaction types.

RxnCLF is a self-supervised learning framework that represents chemical reactions as unified graphs (condensed reaction graphs) to better capture how molecules transform.

Fardin Afdideh, Fernando Seoane, Farhad Abtahi

Post-training adaptation is fragmented across many techniques—this taxonomy provides a unified vocabulary to describe, compare, and govern how models are modified after training, essential for tracking what changes have been made to deployed systems.

This survey creates a comprehensive framework for understanding how trained AI models are modified after initial training. It organizes 50+ adaptation techniques (like fine-tuning, retrieval augmentation, and model editing) into a six-dimensional taxonomy, clarifying confusing terminology and showing how these methods work together in real deployments.

ZhiYan Hou, Xinyu Tang, Hongyan An et al.

Adaptive weighting of token-level supervision based on divergence sequences improves reasoning model training—the timing and context of prediction errors matter more than their magnitude alone.

This paper improves on-policy self-distillation for reasoning models by making supervision weights adaptive to how prediction errors evolve during generation. Instead of treating all mismatches between teacher and student equally, DASH uses gates to weight supervision based on local divergence patterns, improving math reasoning across model sizes without extra computation.

Elad Yoshai, Natan T. Shaked

Per-feature gating based on distribution distance enables better control in unpaired image translation, letting you preserve important structures while achieving realistic style changes without retraining.

PRISM is a new method for changing images from one style to another (like turning day photos into night) without paired training examples. Instead of using a single global control value, it learns a per-feature gate based on how far each image feature is from the target style, allowing precise control over what changes and what stays the same.

Yixiong Xiao, Congxi Xiao, Jingbo Zhou

RAG can boost time series forecasting by retrieving similar past sequences and intelligently fusing them into the model, rather than just appending them like in language models.

This paper adapts retrieval-augmented generation (RAG) to time series forecasting by retrieving similar historical sequences and using special tokens to blend them with input data.

Alperen Kenan, Paul Bremner, Manuel Giuliani

By incorporating force, timing, and multi-segment trajectory handling into standard learning-from-demonstration techniques, robots can generate handwriting that humans perceive as significantly more natural and human-like.

This paper presents a framework for teaching robots to write letters by learning from human handwriting demonstrations. The approach combines force and timing data with trajectory learning to generate robot motions that feel natural to humans. A user study confirmed the generated handwriting was perceived as 71.5% human-like, with the dataset released openly for future research.

Zishan Xu, Zhiyuan Yao, Yuxin Chen et al.

Training agents with internal world rehearsal—where the agent simulates environment responses—is more efficient than external environment interaction and enables better generalization across tool-use tasks.

EnvACE trains AI agents to use tools by having them internally simulate environment responses instead of relying on external simulators. The agent learns to both act and play the environment's role, building an internal world model that predicts what happens after each action. This approach reduces the need for expensive external environments while improving performance on tool-use benchmarks.

Damien Sileo, Valentin Lacombe, Dimitri Kachler

Procedural datasets for training reasoning models need careful design beyond correctness: difficulty calibration, compact targets, and rigorous auditing (combining model review, human judgment, and testing) significantly impact training utility.

This paper introduces Reasoning Core, a collection of 50 procedural generators that create verifiable reasoning problems across diverse domains (math, logic, planning, code, etc.). The authors compare their dataset with three alternatives using completion-supervised fine-tuning on 3B models, showing Reasoning Core achieves better performance on reasoning benchmarks.

Indraneil Paul, Falko Helm, Goran Glavaš et al.

Training language models on code contexts that span multiple related files—rather than isolated snippets—dramatically improves their ability to handle long contexts and understand complex codebases, even when this data makes up a small fraction of total training.

OctoLong is a pipeline that creates long, dependency-rich code contexts by automatically retrieving related code files across repositories.

Yinghui He, Ling Yang, Jiarui Liu et al.

Models struggle to switch between different reasoning skills in long tasks, and explicitly training them to recognize and predict which skill to use at each step dramatically improves their ability to handle complex multi-step reasoning.

This paper introduces Skill Entropy, a metric for measuring how difficult it is for language models to switch between different reasoning skills (like math, planning, coding) in long multi-step tasks. The authors create Skill²-Bench, a benchmark with 558 skills across 9 domains, and show that current models struggle more on tasks requiring frequent skill switches.

Ayoub Kirouane, Christos Petrocheilos

Domain-specific fine-tuning dramatically improves retrieval and generation for underrepresented languages: a modest 1B embedder outperforms multilingual models when trained on just 65K in-domain Greek examples, showing that language adaptation is more important than model size for specialized tasks.

This paper adapts NVIDIA's Nemotron retrieval system for Modern Greek across legal, energy, financial, and medical domains. The authors mine Greek corpora, train specialized retrieval models, and create HERA—the first large-scale Greek RAG benchmark.

Devender Singh

An optimizer's implicit bias toward low-rank solutions depends on whether it respects the gauge symmetry of the loss—a mathematical property independent of the loss value itself.

This paper reveals why different optimizers find different solutions to the same problem: gradient descent implicitly prefers low-rank solutions due to gauge symmetry, while Adam does not. The key difference is whether an optimizer is 'gauge-equivariant'—able to respect the mathematical symmetry of the loss function.

Aniri, Jinhe Bi, Peng Liao et al.

Modality imbalance (text drowning out vision) is a real bottleneck in multimodal reasoning. By detecting when visual input is being ignored and training only on those critical tokens, you can make self-distillation much more effective.

This paper identifies and addresses modality imbalance in multimodal language models—where text dominates over visual information during reasoning. OPD-V uses positive and negative teacher models with modified images to detect when the model isn't properly using visual input, then selectively applies self-distillation only on tokens where visual information matters most.

Adel Javanmard, David P. Woodruff, Vahab Mirrokni

SSTQ enables federated learning systems to compress communications to logarithmic bit-widths while guaranteeing local differential privacy—a significant improvement over prior methods that suffered from dimension-dependent variance.

This paper introduces SSTQ, a privacy-preserving vector quantization method for distributed learning that compresses high-dimensional data into fewer bits while protecting individual privacy.

Yuezhang Peng, Yuxin Liu, Changfeng Gao et al.

Framing spoken language understanding as structured function calling—rather than slot-filling—lets audio language models generalize to new tasks without retraining, similar to how code models handle function calls.

This paper introduces Spoken Function Calling (SFC), a new way to understand spoken language that treats semantic extraction like function calls with structured definitions.

Jai Malegaonkar, Rohan Patil, Henrik I. Christensen

Exploration bonuses and memory architectures aren't independent tools—their effectiveness depends entirely on reward structure.

This paper reveals how reward structure fundamentally shapes whether exploration bonuses help or hurt reinforcement learning agents.

Changle Qu, Sunhao Dai, Hengyi Cai et al.

Turn-level hindsight feedback with cross-horizon validation provides denser, more reliable training signals for tool-using AI agents than traditional trajectory-level approaches, improving learning efficiency in long-horizon reasoning tasks.

TurnSight improves how language models learn to use tools by providing better feedback during training. Instead of judging entire sequences at once, it gives turn-by-turn (step-by-step) feedback based on what actually happened, using multiple future perspectives to decide which feedback is most reliable. This helps models learn faster and more accurately in complex multi-step tasks.

Jinhe Bi, Chennan Zhou, Zengjie Jin et al.

Failed expert trajectories are learning gold: teaching models to reflect on why attempts failed is often easier than solving hard problems from scratch, and this reflective skill transfers to direct problem-solving.

ReflectRL shows that when expert models fail on hard problems, their failed trajectories aren't wasted—they're valuable for learning. The method trains models to reflect on these flawed attempts, then transfers that reflective reasoning back to solving problems directly. This lightweight approach works across multiple benchmarks and training methods with minimal computational cost.

Yordan Raykov, Rodrigo Veiga

By treating forward-policy training as a geometric optimization problem, you can leverage the target distribution's structure (factorization, locality) to compute better gradient updates, leading to faster convergence and better exploration in GFlowNets.

This paper reformulates how to train the forward policy in GFlowNets (a framework for sampling from complex distributions) using information geometry.

Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina et al.

Organizing tasks into separate optimization paths with independent adapters outperforms using a single shared adapter—even with the same total parameters—because it reduces task interference and preserves positive transfer between compatible tasks.

This paper addresses a key problem in fine-tuning large language models: when you try to adapt a single shared adapter to many different tasks, they interfere with each other, hurting performance. The authors propose automatically organizing tasks into groups and sequences, then training separate adapters for compatible tasks instead of forcing everything through one shared adapter.

Haomin Qi, Xingliang Wang, Xuanqi Gao et al.

By mining repository history and reconstructing code states, you can generate verified, executable coding tasks at scale—reducing the manual effort of creating training data for code-generation agents while maintaining realistic development scenarios.

Change2Task automatically converts pull requests from repository history into executable coding tasks for training AI agents.

Kangning Zhang, Yixing Li, Shuai Shao et al.

By attributing teacher corrections to visual evidence through counterfactual intervention, VAD extracts purer visual supervision signals for distillation, outperforming naive privileged-view supervision that mixes visual and linguistic signals.

This paper addresses a key problem in multimodal distillation: when a teacher corrects a student's mistakes, it's unclear how much of that correction comes from visual evidence versus linguistic priors. VAD solves this by using counterfactual reasoning—removing visual evidence and measuring how the teacher's predictions change—to isolate visually-grounded corrections.

Jiawei Xu, Minghui Liu, Juzheng Zhang et al.

By parameterizing the KL penalty weight (β) in self-distillation, you can trade off stability and teacher guidance more flexibly, and implement the optimal policy through simple logit mixing rather than expensive RL optimization.

This paper improves on-policy self-distillation for reasoning models by introducing β-OPSD, which treats the KL penalty weight as a tunable parameter rather than a fixed value. The method derives an optimal policy as a geometric blend between a reference policy and a teacher model, then implements this efficiently through logit mixing during distillation.

Junlin Yang, Che Jiang, Yu Fu et al.

You can train smaller models to autonomously improve ML code and configurations by teaching them to apply specific program-evolution operators in a loop, enabling practical AI-assisted ML engineering without massive compute.

This paper introduces Frontis-MA1, a 35B AI model trained to improve machine learning engineering tasks through recursive self-improvement. The system uses four core operators (Draft, Improve, Debug, Crossover) to iteratively refine ML solutions, achieving 71% performance on benchmarks—comparable to much larger models—while running on a single GPU with 12GB memory.

Shentong Mo, Yatao Bian

Physical consistency constraints alone can guide better 3D structure prediction than supervised coordinate matching, enabling practical structure modeling when ground-truth labels are unavailable.

APO is an unsupervised method for predicting 3D atomic structures (crystals, proteins) without needing labeled reference structures. It uses a dual-reward system—one that identifies physically realistic configurations through statistical analysis, another that enforces thermodynamic stability—allowing models to self-correct during training.

Neelam Akula, Surbhi Kumar, Murat Kantarcioglu et al.

Node classification helps link prediction consistently, but link prediction rarely helps node classification unless the graph structure is very predictable—use graph homophily as a signal for whether to share model weights across these tasks.

This paper investigates how knowledge transfers between node classification and link prediction tasks on the same graph. The authors establish a standardized evaluation protocol to eliminate unfair comparisons, then show that transfer works well in one direction (NC→LP) but is unreliable in the other (LP→NC).

Andreu Casamayor-Segarra, Vicent Ahuir, Antonio Molina-Marco et al.

Domain-specific Spanish models combined with ICE's automatic relabeling can detect mental health risks earlier and more reliably than general approaches, addressing a critical gap in non-English mental health AI tools.

This paper tackles mental health screening in Spanish by creating specialized language models and a new method called Incremental Context Expansion (ICE) that automatically identifies when enough social media messages accumulate to signal a mental health disorder. The approach reduces the time needed to detect problems while maintaining accuracy, with all models made publicly available.

Hongyu Chen, Liang Lin, Guangrun Wang

Models can learn to self-verify and adaptively allocate test-time compute by treating their own correctness predictions as a control signal, eliminating the need for external verifiers while improving efficiency.

SVR teaches language models to verify their own answers during inference and decide when to stop refining, without needing external feedback. The model learns to produce solutions with confidence scores and correctness verdicts, continuing to refine only when uncertain.

Yecheng Wu, Song Han, Han Cai

You can now use different models to generate training data and provide distillation supervision without performance loss, by filtering out style differences rather than requiring teacher consistency.

This paper addresses a practical problem in training large reasoning models: when the teacher model used for distillation differs from the one that generated the training data, performance often suffers.

Perry Dong, Ron Polonsky, Dorsa Sadigh et al.

When fine-tuning RL policies, pretraining Q-functions on offline data often doesn't help; instead, initializing Q-functions from diverse policy rollouts works better and is simpler.

This paper challenges the conventional wisdom that Q-functions should be pretrained for online RL fine-tuning. The authors show that naive Q-function pretraining often doesn't help because it targets the wrong policy, then propose IPE—a method that uses diverse pretrained policies to bootstrap Q-function learning, achieving 1.26x better performance than standard pretraining.

Raphaël Sourty, Antoine Chaffin, Paulo Roberto Moura Junior et al.

Late-interaction retrieval models generalize better across languages than dense models when trained on translated data, suggesting token-level matching is more effective for multilingual transfer than vector-level matching.

This paper presents open-source recipes for training retrieval models at scale. The authors curate 665M English training pairs and create two 149M-parameter models (DenseOn for dense retrieval, LateOn for late-interaction matching) that achieve state-of-the-art results.

Haolei Xu, Xiaowen Xu, Haiwen Hong et al.

When training smaller models to reason, letting a teacher briefly take over at critical failure points—rather than forcing the student to learn from its own mistakes—significantly improves both efficiency and accuracy.

This paper addresses a key failure mode in on-policy distillation where students get stuck in wrong reasoning paths. The authors propose Relay-OPD, which detects when a student diverges from the teacher's approach and temporarily hands off to the teacher to correct course, then resumes student training.

Tom Saliencro, Rohan Desai, Priya Nair et al.

You can route tokens to variable numbers of experts based on per-token confidence signals from the router itself, improving efficiency and performance without adding parameters or complexity.

This paper proposes CARE, a smarter way to route tokens through expert networks in fine-tuned language models. Instead of sending every token to the same fixed number of experts, CARE uses the model's own uncertainty signals to decide how many experts each token needs—easy tokens get fewer experts, hard tokens get more.

Gaspard Lambrechts, Adrien Bolland, Daniel Ebi et al.

Using additional supervision (privileged information) during training can significantly improve how RL agents learn world models, but the way you guide the learning matters—latent guidance works better than previous approaches.

This paper improves model-based reinforcement learning by using extra information during training to learn better world models. The authors identify a problem with how previous asymmetric methods handle privileged information and propose Reinformed Dreamer, which uses latent guidance to learn more useful representations. Tests show it outperforms the baseline Dreamer algorithm more consistently.

Siyu Xia, Chenheng Zhang, Yanting Wu et al.

By combining retrieval-based and learned memory with automatic routing, UniMem lets AI agents efficiently handle evolving task streams without manual task labels or fixed memory constraints.

UniMem is a memory management system for AI agents that balances two types of memory: episodic (retrieval-based) for new or rare tasks and parametric (learned weights) for recurring patterns. Using learnable routing tokens, it automatically decides which memory type to use for each task, avoiding the need for explicit task boundaries and enabling efficient learning from continuous task streams.

Zandi Eberstadt

Instruction-tuned LLMs show less syntactic convergence to human dialogue partners than pretrained models, indicating that alignment training may inadvertently reduce a natural form of linguistic adaptation that humans exhibit.

This study examines whether large language models adapt their grammar to match human speakers in dialogue, similar to how humans naturally do.

Yuan Yin, Elias Ramzi, Marc Lafon et al.

Training driving policies directly from egocentric camera images at scale is now practical—eliminating the need for privileged observations during training while maintaining competitive performance and improving real-world transfer.

Pictura is a GPU-accelerated driving simulator that renders camera views for each agent in real-time, enabling self-play training of driving policies directly from images rather than privileged vectorized data.

Neta Shaul, Chao Liu, Arash Vahdat et al.

PDD enables fast video/image generation (4-8 steps instead of 50+) with simpler training than current methods, while actually improving output diversity—making it practical for scaling generative models to production.

This paper introduces Parallel Decoding Distillation (PDD), a method to speed up image and video generation from diffusion models by predicting multiple denoising steps at once instead of one at a time.

Wenzhi Zhong, Edward Milsom, Michael Murray

Using spectral norm geometry in SAM's perturbation step combined with Muon optimization leads to better generalization on vision models—a practical improvement for training robust neural networks.

This paper improves Sharpness-Aware Minimization (SAM), a training technique that helps models generalize better, by using matrix-aware geometry. The authors propose using spectral norm-based perturbations for hidden-layer weights and combine this with the Muon optimizer, achieving better results on ImageNet with ViT and ResNet models.

Bingnan Li, Haozhe Wang, Haozhong Xiong et al.

When distilling guided diffusion models, directly matching teacher and student predictions can backfire if they have asymmetric access to information—you need to supervise different prediction branches separately to avoid antagonistic error dynamics.

This paper identifies a critical flaw in how diffusion models are distilled when using classifier-free guidance (CFG). The authors show that naive matching of teacher and student predictions can fail when the teacher has access to information the student doesn't, causing errors in different prediction branches to work against each other.

Tianyi Men, Zhuoran Jin, Kang Liu et al.

Long-horizon planning improves most when agents learn explicit world models during pre-training, and multi-teacher distillation works better than single-teacher methods for integrating diverse planning strategies across tasks.

This paper systematically studies how AI agents learn long-horizon planning through three stages: pre-training (what data helps), post-training (GRPO vs OPD methods), and multi-teacher integration (MOPD).

Zhen Huang, Yikun Wang, Shijie Xia et al.

Instead of applying uniform data processing rules, adapting the cleaning strategy per example—deciding what operation each piece of data needs—improves LLM pretraining efficiency and downstream performance.

DataOrchestra is a framework that customizes data processing for each example in pretraining, rather than applying one fixed strategy to all data. An orchestrator decides whether to drop, keep, or clean each data chunk, and if cleaning is needed, selects specific operations like editing or rewriting.

Zeyu Zhang, Xue Li, Iacer Calixto et al.

When adapting language models for entity matching, the model's pretraining variant (embedding-focused vs. general) matters more than raw size, and architectural advantages depend heavily on whether you're dealing with distribution shift or cross-dataset scenarios.

This paper systematically compares how language models perform at entity matching (identifying records referring to the same real-world entity) across different architectures, model variants, and sizes.

Francisco Mena, Dino Ienco, Roberto Interdonato et al.

Co-learning between modalities is more robust than fusion-based approaches when dealing with unpredictable missing modalities at inference time, with different strategies optimal for mild versus extreme missing conditions.

This paper tackles multi-modal classification when some data sources are unavailable at inference time—a common real-world problem from sensor failures or privacy constraints. Instead of designing better fusion methods, the authors use co-learning where modalities teach each other.