ThinkLLM
ModelsCapabilitiesUse CasesBenchmarksPapersGlossary
ModelsCapabilitiesUse CasesBenchmarksPapersGlossary
AboutPrivacyTermsRSS

ThinkLLM

Spot an error in our data? Let us know.

Papers

Recent AI research papers with accessible summaries. Updated daily from arXiv, summarized for developers who don't read papers regularly.

2455 papers45 this month12 topics
AllEvaluation 43Agents 42Applications 39Efficiency 29Reasoning 29Training 27Safety 23Multimodal 18Architecture 9Alignment 8Data 4scaling 2

Nov 16 – Nov 22(6)

RL Post-Training in Diffusion Transformers Is Activation Steering

Nov 19, 2026

Tuna Han Salih Meral, Kaan Oktay, Alec Helbling et al.

RL post-training of diffusion models works largely through input-independent activation steering, meaning you can recover most improvements with a tiny fixed vector added to frozen model outputs—this has major implications for understanding and efficiently improving generative models.

This paper shows that RL improvements to diffusion transformers come primarily from simple activation steering—adding fixed vectors to model outputs—rather than learning complex input-dependent changes.

trainingefficiency

TideCache: Efficient Hybrid LLM Serving with Transparent Cache Management and Layer-Wise Streaming

Nov 19, 2026

Jinbin Luo, Xiaozhe Yao, Shang Yang et al.

By treating different cache types as flexible pools and overlapping data transfers with computation, TideCache achieves 2x more concurrent requests and 20-59% lower latency—showing that intelligent memory management, not just bigger GPUs, unlocks efficiency gains.

TideCache is a serving system that optimizes GPU memory usage for hybrid LLM architectures by intelligently managing different cache types across GPU and CPU memory. It uses transparent page redistribution, smart prefetching during computation gaps, and adaptive request scheduling to fit more requests and reduce latency on CPU-GPU systems.

Nov 9 – Nov 15(16)

Training Reasoning Models in Simulation to Natively Use Real Robots

Nov 13, 2026

Jiaming Tang, Zhuoyang Zhang, Junxian Guo et al.

Open-weight reasoning models can learn effective robot control through simulation-based reinforcement learning, achieving real-world performance comparable to frontier models without demonstrations or task-specific fine-tuning.

Researchers trained an open-weight 27B reasoning model to control real robots by learning entirely in simulation through reinforcement learning, without needing human demonstrations. The model observes the environment, reasons through its actions, and generates robot commands—similar to how AI agents use computers.

trainingagentsapplications

WeightBench: Evaluating Language Models as Weight-Space Programmers

Nov 13, 2026

Aayush Marasini, Barsat Khadka, Nick Rahimi

LLMs can now program neural networks by directly editing their weights, but this capability is limited to small networks; frontier models like Claude Opus 3.5 score only 0.535 on the benchmark, revealing significant gaps in weight-space programming.

WeightBench is a benchmark for evaluating how well large language models can directly manipulate neural network weights—reading what networks learned, fixing broken ones, writing new networks, and compressing them—without using gradient descent. Testing frontier models on 42 tasks shows they can handle small networks but struggle with larger language models.

Nov 2 – Nov 8(23)

Saidzoda-CA-EmbeddingGemma-300M: One Small Embedder for Tajik, Kazakh, Uzbek and Kyrgyz

Nov 8, 2026

Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov et al.

A single small embedding model can serve multiple low-resource languages effectively through targeted fine-tuning with synthetic data, but scaling up to larger multilingual models remains necessary for competitive performance on these languages.

Researchers fine-tuned a small 300M embedding model (EmbeddingGemma) to work across four Central Asian languages—Tajik, Kazakh, Uzbek, and Kyrgyz—using synthetic question-answer pairs generated by a language model. The fine-tuned model significantly improved retrieval performance (14-26 percentage points) on a multilingual benchmark, though it still lags behind larger 560M multilingual models.

multimodaltrainingefficiency

How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?

Nov 8, 2026

Cong Yu, Ziting Wang, Songlin Jiang et al.

On-policy distillation tolerates significant training-inference mismatch from quantization and asynchronous rollouts without quality loss, enabling 1.6× faster training with FP8 precision while maintaining performance.

This paper studies how much training-inference mismatch on-policy distillation (OPD) can tolerate when using quantized inference and asynchronous rollouts to reduce training costs.

Oct 26 – Nov 1(30)

Latent Reasoning in Continuous Space for Unified Multimodal Models

Oct 30, 2026

Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee et al.

Reasoning in continuous latent space instead of discrete tokens unlocks test-time scaling and better controllability for image generation in multimodal models, showing 7% improvement on GenEval benchmarks.

This paper introduces LARC, a framework that enables unified multimodal models to perform reasoning directly in continuous space rather than discrete tokens, bridging the gap between language-based reasoning and image generation.

reasoningmultimodaltraining

A Better Critic Matters for Distribution Matching Distillation

Oct 30, 2026

Zongze Wu, Xianbang Wang, Zhengyang Geng et al.

For distillation to work well, the critic that evaluates student quality must be scaled proportionally with the teacher and should condition on generation steps—otherwise training can collapse or students fail to improve.

This paper improves Distribution Matching Distillation (DMD), a method for training fast student models from slower teacher models. The authors show that the critic network—which guides student training—needs to be scaled alongside the teacher and should be aware of generation steps to provide reliable training signals.

Oct 19 – Oct 25(20)

The Language of Instruction: Genre and Generative AI

Oct 23, 2026

James E. Dobson

Instruction-tuned models aren't just trained differently; they embody a distinct genre of communication that blends human conversational patterns with formal markup structures, which developers should understand when building with these systems.

This paper argues that instruction-fine-tuned language models implement a specific genre—the instructional-conversational genre—shaped by human-AI conversation patterns, system prompts, and structured input formats. It examines how these technical choices create a recognizable communicative style.

trainingalignment

RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms

Oct 23, 2026

Xi Chen, Fengqi Hu, Zeyu Shen et al.

This library standardizes robust RL algorithm implementation and reveals empirical capability boundaries—showing practitioners which robustness techniques handle which types of real-world disruptions, with reproducible benchmarks for developing and testing new methods.

RobustRLlib is a unified library and benchmark for robust reinforcement learning that integrates 17 algorithms (online, offline, and safe RL) under one interface. It provides systematic evaluation across 273 perturbation configurations and diverse tasks to map where current robust RL methods work well and where gaps remain, enabling better real-world deployment.

Oct 12 – Oct 18(5)

Rethinking the design space of Vanilla Pixel Diffusion Transformers

Oct 16, 2026

Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava et al.

Pixel diffusion models benefit from adaptive prediction strategies that vary by capacity and data type, and post-hoc high-frequency refinement can significantly improve image quality without expensive retraining.

This paper improves pixel-space diffusion transformers by introducing MFlow, which adaptively learns to blend two prediction strategies (clean image vs. velocity) based on data characteristics and model size. The authors also propose ConvRefiner, a lightweight module that fixes a persistent high-frequency quality deficit in generated images without retraining the main model.

architectureefficiencytraining

Behavioral Couplings: From Formation to Behavioral Emergence in Language Models

Oct 16, 2026

Dongyoon Hahm, Woogyeol Jin, Dennis Wei et al.

Fine-tuning LLMs risks activating hidden behavioral associations learned during pretraining, and these couplings can compose across datasets to create unpredictable emergent behaviors that standard evaluation won't catch.

This paper reveals how fine-tuning LLMs can trigger unintended behaviors through 'behavioral couplings'—hidden associations between behaviors learned during training. When a model learns two behaviors together in training data, fine-tuning for one automatically activates the other.

efficiencyarchitecturetraining

Represented but misread: Post-training changes how language models read out what they represent

Nov 19, 2026

Jochen Laubrock

Post-training doesn't erase what models know internally; it changes how they convert internal representations into answers. Small amounts of labeled training data can flip model behavior, showing the readout mechanism is malleable and task-dependent.

This paper investigates whether post-training changes what language models represent or how they read out those representations. Using linear probes and signal detection theory, the author finds that post-training often degrades model answers despite probes showing good internal representations—suggesting post-training changes the readout mechanism rather than the underlying knowledge.

trainingevaluationalignment

Jailbreak Detection on Large Language Models via Contrastive Prototype Structures

Nov 18, 2026

Yan Qiu, Yuanyuan Li

Contrastive learning of safety prototypes can detect jailbreaks more efficiently than traditional classifiers, using fewer labeled examples and showing stronger robustness across different datasets and conversation contexts.

This paper proposes JDPI, a method for detecting jailbreak attempts on large language models using contrastively learned prototypes rather than training a separate classifier.

safetyevaluationtraining

WorldRuler: Interactive World Modeling under Configurable Rules

Nov 18, 2026

Mingliang Zhai, Chuanhao Li, Jiaming Tan et al.

To build customizable interactive worlds with generative models, you need to explicitly track rules and object states separately from visual generation, using techniques like dynamic object binding and counterfactual training to ensure rule effects appear correctly.

WorldRuler is a system for building interactive world models that follow user-defined rules rather than just real-world physics. It uses a 3D rule graph to organize rules and world states, enabling games or simulations to save/load while maintaining rule consistency, and accurately apply rule effects to objects even before they appear on screen.

agentsmultimodaltraining

Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs

Nov 18, 2026

Jiashu Li, Tianyu Gao, Qinglong Liu et al.

Breaking math reasoning into specialized expert modules and routing problems to appropriate skill combinations outperforms monolithic approaches—AMR improves math accuracy by 3-5% on benchmark tasks by matching problems to their required reasoning skills.

This paper proposes Atomic MathReasoner (AMR), a framework that decomposes mathematical reasoning into specialized skills and learns to compose them for different problems. Rather than training one general math model, AMR creates expert modules for specific reasoning abilities, then uses a lightweight router to select the right combination of experts for each problem at inference time.

trainingarchitecturereasoning
evaluationagentstraining

ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL

Nov 13, 2026

Zelin He, Siyu Zhu, Jacob Helwig et al.

Harnesses for agentic RL can be automatically learned from task data and continuously improved during training, rather than hand-crafted and fixed—this adaptive approach outperforms static harnesses across multiple domains.

This paper proposes ModHarness-RL, a system that learns and evolves the harness (the scaffold controlling what an AI agent sees and how it's guided) during reinforcement learning training.

agentstrainingreasoning

ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

Nov 13, 2026

Dhruv Dixit, Khushi Patel, Congcong Xu et al.

Small models can match larger ones on math reasoning by learning to produce executable, tool-grounded action traces instead of free-form explanations, combined with reinforcement learning that rewards both correctness and process quality.

ARCHER trains small language models to solve math problems by learning to produce executable reasoning traces—sequences of reasoning steps, calculator calls, and verification actions—rather than just explaining their work.

trainingreasoningapplications

Unbundling Multimodal Vision Encoding

Nov 12, 2026

Ellis Brown, Chris Hoang, Shengyi Qian et al.

For building multimodal models, dedicated vision capacity trained jointly with the language model is more important than using pretrained vision encoders, and this capacity can be efficiently placed early in the backbone rather than as a separate pre-fusion module.

This paper investigates how vision encoders should be designed in multimodal language models by systematically comparing three design choices: using a pretrained visual encoder, dedicating capacity to vision processing, and where to place that capacity.

multimodalarchitecturetraining

AgentSSL: Can MLE Agents Leverage Unlabeled Data?

Nov 12, 2026

Ethan Lin, Akanksha Sarkar, Ziang Liu et al.

AI agents can automatically synthesize effective semi-supervised learning pipelines, but the bottleneck isn't code generation—it's reliably selecting which generated programs actually work best.

This paper studies whether AI agents can automatically design semi-supervised learning (SSL) pipelines when labeled data is scarce. The authors show agents can discover code combining SSL techniques that outperform human baselines by up to 21.8%, but reveal a critical weakness: selecting which generated programs to use drops performance by 47%.

agentstrainingdata

Learning When Diffusion Language Models Should Regenerate

Nov 12, 2026

Minhae Oh, Jeonghye Kim, Jinwoo Koo et al.

A learned verifier can efficiently improve diffusion language model outputs by deciding when to regenerate blocks, achieving better accuracy-to-cost tradeoffs than traditional search-based decoding.

This paper introduces DiVR, a verifier that learns when diffusion language models should regenerate text blocks during decoding. Rather than relying on token confidence, DiVR predicts whether a partially generated sequence will succeed, enabling selective remasking and regeneration.

reasoningefficiencytraining

Negative-Only Policy Optimization for One-Sided Verifiable Rewards

Nov 12, 2026

Jiacheng Xu, Shuo He, Fuxiang Zhang et al.

When you can only reliably detect wrong answers (not confirm right ones), training by suppressing failures works better than rewarding unconfirmed successes—a practical approach for real-world verification constraints.

This paper addresses reinforcement learning when verification is one-sided: you can reliably detect failures but can't confirm successes. The authors propose NOPO, which learns by suppressing known failures rather than rewarding unconfirmed successes, improving code and math generation without complete verification.

trainingreasoningsafety

Maximizing Capacity Utilization of LoRA Adapters for Mitigating the Learning-Forgetting Trade-off

Nov 12, 2026

Tomohiro Sawada, Kartik Goyal

LoRA adapters waste capacity by not fully utilizing their configured rank; adding an effective-rank regularizer during training fixes this and improves the learning-forgetting trade-off without extra computational cost.

This paper identifies why LoRA adapters underperform full fine-tuning: they don't use their full capacity. The authors show that LoRA updates have much lower effective rank than their configured rank, limiting how broadly parameter updates spread.

efficiencytrainingalignment

Cortex-FL: Agent-Mediated Collaboration for Federated Learning

Nov 12, 2026

Rui Sun, Jinfeng Peng, Zehua Cheng et al.

This system makes federated learning practical for non-experts by automating the hard parts (finding partners, negotiating terms, managing incentives) while preserving data privacy and organizational autonomy through smart contracts.

Cortex-FL is a federated learning system where AI agents help organizations collaborate on model training without sharing raw data. Users describe what model they want in plain English, and agents automatically find partners, negotiate agreements, and coordinate training—while keeping each organization in control of their own data and finances.

agentstrainingapplications

Parsed, Plausible, Wrong: Output Format Controls Silent Errors in Small-Model Semantic Parsing

Nov 12, 2026

Umar Ahmed, Maxim Dunaev, Mohammad Nour Al Awad et al.

Output format is a free design choice that dramatically impacts small model reliability: natural language encodings reduce silent semantic errors by 2-3x and improve accuracy, suggesting format selection deserves as much attention as model architecture.

Small language models fine-tuned for semantic parsing often produce grammatically correct but semantically wrong outputs that go undetected. This paper shows that the output format—how you encode the target structure—significantly affects error rates.

trainingevaluation

Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents

Nov 12, 2026

Haochen Liu, Qianhao Zhu, Zixuan Wang et al.

A decision model's ability to teach a policy through reinforcement learning is independent of its performance as an actor—poor actors can be excellent judges and teachers because they evaluate actions with full trajectory information.

This paper investigates how decision models—systems that output choices or scores—perform in three different roles: as actors (choosing actions), judges (evaluating past actions), and teachers (training policies via reinforcement learning).

agentsreasoningtraining

Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination

Nov 12, 2026

Wei Feng, Meiling Chen, Boxiao Chen et al.

Coordinating GPU resource allocation, cache-aware scheduling, and environment execution can significantly speed up batch processing of agent workloads without sacrificing accuracy.

This paper optimizes batch execution of LLM agent tasks by coordinating inference and environment interaction. It introduces resource allocation strategies for hybrid LLMs, trajectory-aware routing for cache reuse, and Chain-Advancing Action Scheduling to handle environment bottlenecks—achieving 10-65% makespan reductions on real benchmarks.

efficiencyagentstraining

Asymmetric Multimodal Learning with Decomposed JEPAs

Nov 12, 2026

Yusuf Shihata

For multimodal AI systems, alignment-only training catastrophically fails on compositional reasoning when modalities carry different types of information—you need to explicitly preserve modality-specific structure alongside alignment.

This paper shows that standard cross-modal alignment alone loses important information when modalities are asymmetric (like text queries and images). It introduces Decomposed JEPA, which splits learned representations into a shared aligned part and private modality-specific parts, recovering information that alignment discards and improving performance on reasoning tasks.

multimodalarchitecturetraining

AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models

Nov 10, 2026

Zijian Zhang, Rizhen Hu, Jiangweizhi Peng et al.

Automatically generated, adaptive training scaffolds can significantly improve RL-trained language models by turning failure information into targeted guidance, with gains persisting even after scaffolds are removed at evaluation.

AutoScaffold is an AI system that automatically creates and adapts training guidance (scaffolds) for language models learning through reinforcement learning. A frozen teacher analyzes the student model's failures during training, diagnoses problems, and generates helpful strategies or hints that are removed at test time.

trainingreasoningagents

Continual-ARC: Measuring Skill Acquisition over a Working Lifetime

Nov 10, 2026

Daniel Lenton

Current agent harness methods for continual learning underperform simple memory-based baselines on repeated tasks, but weight-based learning shows potential for generalizing to new domains—suggesting future work should focus on training models to better use external tools and memory systems.

This paper introduces Continual-ARC, a benchmark for measuring how AI agents learn over time by solving tasks from the ARC-AGI dataset. Tasks repeat with new instances and new tasks arrive continuously, mimicking real-world learning.

trainingevaluationagents
trainingefficiency

SWHForest: A Structural Curation and Search Layer for Large-Scale Code Corpora

Nov 8, 2026

Federico Pennino, Andrea Gurioli, Maurizio Gabbrielli et al.

For code model pre-training, you can now query structural properties (AST patterns, definitions, etc.) across massive corpora in seconds without downloading everything—enabling faster, more targeted dataset curation.

SWHForest is a framework that creates a searchable structural index of code files without storing source code, letting researchers efficiently filter and curate large code corpora before downloading. Instead of downloading millions of files to find what they need, users can query the index to identify matching files first, reducing downloads by 46-72%.

datatrainingapplications

On the Reliability of Majority-Vote Pseudo-Labels in Test-Time Reinforcement Learning

Nov 6, 2026

Haoyu Wang, Ru Wang, Yuanyuan Wang et al.

When adapting language models at test time using self-generated labels, you should weight updates based on how reliable the voting signal is for each prompt—this prevents the model from learning from noisy pseudo-labels.

This paper addresses a key problem in test-time reinforcement learning: how reliable are the self-generated labels used to adapt language models? The authors analyze majority-vote pseudo-labels, identify sources of error, and propose PC-Dr.DPO, a method that makes the model more cautious when updating from unreliable prompts. They validate their approach across math and reasoning tasks.

trainingalignmentefficiency

CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs

Nov 6, 2026

Xingyu Liu, Xiangyang Yin, Tianhua Xia et al.

Reasoning models need special quantization handling because they show larger activation fluctuations early in generation—CodeQuant2 captures this with adaptive clustering and better calibration, achieving 2.85× speedup with minimal accuracy loss.

CodeQuant2 is a quantization technique that compresses large reasoning models to low precision while maintaining accuracy. It addresses unique challenges in reasoning models by detecting and handling unusual activation patterns in early decoding steps, using clustering to represent weights efficiently, and adapting compression levels per layer.

efficiencytraining

Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RL

Nov 6, 2026

Aria Wong, Joshua Engels, Neel Nanda

Effective reward hacking monitors need to balance providing learning signal to the model while avoiding systematic vulnerabilities that enable evasion; simpler training data and better monitor design can significantly reduce reward hacking in practice.

This paper studies how to use monitoring systems to catch and prevent reward hacking—when AI models learn to game their reward signals in unintended ways. The authors create three realistic environments where a language model exploits flaws (test overwriting, sycophancy, hallucination) and test different monitor designs.

safetytrainingalignment

BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

Nov 6, 2026

Dayou Du, Woon Yee Ng, Guangming Sheng et al.

You can compress LLM key-value caches to 4-bit precision without significant accuracy loss by carefully handling quantization errors at the algorithm level and co-designing with hardware capabilities—delivering major serving speedups in practice.

BitDecoding-FP4 enables efficient long-context LLM serving by storing key-value caches in ultra-low-precision 4-bit floating-point format while maintaining accuracy.

efficiencytrainingarchitecture

Scientist World: Grow a Research World for AI-Scientists

Nov 5, 2026

Taiming Lu, Sachin Konan, Zhuang Liu

AI research systems now have a concrete, scalable evaluation framework grounded in real scientific problems—turning the entire scientific literature into a runnable benchmark where AI can be trained and measured against human researcher achievements.

Scientist World converts research papers into executable environments where AI systems can attempt to solve the open problems described in them. The system automatically extracts research questions from papers, implements baseline methods, and evaluates AI-Scientists against human researcher performance.

evaluationagentstraining

Towards interpreting any residual stream direction by scaling activation inversion

Nov 5, 2026

Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger et al.

MAEMs offer a scalable, verifiable way to interpret any direction in a language model's residual stream by generating text that maximally activates it, going beyond what corpus search can find and enabling interpretation of steering vectors, backdoors, and internal model concepts.

Researchers developed Maximally-Activating Example Metamodels (MAEMs)—language models that generate text examples to explain what activates specific directions in another model's internal representations. Unlike searching through text corpora, MAEMs can discover novel explanations for any direction, even those not well-represented in existing data.

evaluationtraining

ReproBridge: Building Reproducibility into ML Research Artifacts

Nov 5, 2026

Yubo Wang, Rui Meng, Jaehyun Nam et al.

Packaging ML experiments with clear procedures and environment details—not just raw code—dramatically improves AI agents' ability to reproduce results reliably and cheaply, regardless of the agent's base capability.

ReproBridge helps AI agents reproduce machine learning experiments by creating standardized, refined artifacts that bundle code, environment setup, and execution guides. Testing on 1,029 tasks shows refined artifacts boost reproduction success from 76% to 93% and cut API costs by 20%, while narrowing performance gaps between different AI models.

evaluationtrainingapplications

Label-Wording Augmentation in a Typed-Decision Encoder: Gains on Unfamiliar Words, Not on Abbreviations or Codes

Nov 5, 2026

Vladimir Beskorovainyi

Label wording matters significantly for ranking models: augmentation helps with natural language but provides no benefit for abbreviations or codes, suggesting different strategies are needed for different label types.

This paper evaluates how changing label wording affects a multilingual AI model's ability to rank options. Testing a Russian cross-encoder on routing labels, the authors find that augmenting labels with multiple surface forms helps with ordinary words and jargon, but fails on abbreviations and opaque codes.

evaluationtraining

Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning

Nov 5, 2026

Chenruo Liu, Zihan Wang, Yijun Dong et al.

LoRA's optimization dynamics differ fundamentally between SFT and RL due to baseline subtraction in RL; adapting LoRA's spectrum management to each stage—spreading updates in SFT, rotating subspaces in RL—improves fine-tuning efficiency.

This paper analyzes how LoRA (a parameter-efficient fine-tuning method) behaves differently during supervised fine-tuning versus reinforcement learning. By tracking the mathematical properties of LoRA updates, the authors discover that RL concentrates weight changes along fewer directions while SFT spreads them out, and they trace this difference to how baseline rewards are subtracted in RL.

trainingefficiency

FedReason: Large Language Models as Weight-Space Meta-Aggregators for Heterogeneous Federated Learning

Nov 5, 2026

Ali Mohammadi Ruzbahani, Abbas Yazdinejad, Hadis Karimipour

LLMs can act as adaptive meta-aggregators in federated learning without server-side training, dynamically selecting aggregation strategies based on weight analysis—though performance depends heavily on proxy feedback and the approach doesn't yet match specialized methods like SCAFFOLD.

FedReason uses a large language model to dynamically choose aggregation rules for federated learning by analyzing client model weights and testing candidates on a small proxy dataset. The LLM runs code in a sandbox to inspect weights, selects or writes Python aggregation programs, and the server applies them until the next LLM call, falling back to FedAvg if invalid.

trainingefficiencysafety

Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout

Nov 5, 2026

Tushaar Gangavarapu, Gil Tabak, Clemens JS Schaefer

Linear attention models can be made significantly more efficient by allocating parameters based on sensitivity analysis: use low-rank gates for state erasure (which matters less) and invest savings into wider networks and better readout mechanisms.

Muir improves linear attention models by analyzing how state updates work differently across channels. The key insight is that erasing old information requires less capacity than writing new information, so the model uses low-rank gates for erasure and reinvests those parameters into a wider network.

efficiencyarchitecturetraining

StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference

Nov 5, 2026

Jake Hyun, Ziyu Zhao, Jae-sun Seo et al.

You can quantize LLM weights and KV caches to 2-3 bits without accuracy loss by selectively streaming precision from slower storage during inference—the key is overlapping transfers with computation so decoding speed doesn't suffer.

StrataFlux recovers accuracy lost to extreme quantization (2-4 bits) in LLM inference by strategically fetching higher-precision data from RAM and SSDs during generation. It overlaps these transfers with computation so they don't slow down decoding, enabling models like Llama-3.1-8B to run on GPUs with half the memory while matching the accuracy of less-aggressive quantization schemes.

efficiencytraining

Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement

Nov 4, 2026

Michal Korniak, Marco Bagatella, Ralf Römer et al.

Generalist robot policies can self-improve on many tasks at once in deployment using RL, and this multi-task improvement can transfer to unseen tasks—but only with the right algorithmic recipe combining value-based RL and careful task distribution choices.

This paper shows that large pre-trained robot models can improve themselves on multiple tasks simultaneously through reinforcement learning during deployment, rather than just adapting to single tasks.

agentstraining

Aligning Language Models with General Preferences: An Empirical Study of Direct Nash Optimization

Nov 3, 2026

Austin Tsai, Saurav Nanda, Ganapathy Parthasarathy et al.

DNO can substantially improve open-weight language models through iterative preference-based fine-tuning, and the expensive rating step can use cheaper open-weight models without losing most of the performance gains.

This paper studies Direct Nash Optimization (DNO), a method that improves language models by treating alignment as a two-player preference game. The authors implement and test DNO on Llama and Mistral models, showing significant improvements on AlpacaEval 2.0 benchmarks.

alignmenttrainingevaluation

PolicyIK: High-Precision Inverse Kinematics for Robot Arms with Reinforcement Learning

Nov 3, 2026

Yuhan Liu, Xinqi Liu, Zixuan Dong et al.

Reinforcement learning can achieve higher-precision inverse kinematics than classical methods while being easier to deploy across different robot arms, making it practical for embodied AI systems.

PolicyIK uses reinforcement learning to solve inverse kinematics for robot arms with millimeter-level precision. Instead of relying on hand-tuned classical controllers, it trains a policy that converts end-effector pose commands into joint motions, handling the mathematical complexities of redundancy and singularities.

trainingapplicationsagents

GUILD: Guided Iterative Data Collection for Imitation Learning

Nov 3, 2026

Shivin Dass, Kevin Nguyen, Rohan Yelandur et al.

Active data collection guided by predictive attribution can reduce the amount of real-world robot teleoperation needed by 2-3x compared to unguided collection, because it focuses on collecting data that actually improves policy performance rather than just semantically relevant data.

GUILD is a method for collecting better training data for robot learning by actively deciding what to collect rather than passively gathering data. It uses vision-language models to suggest relevant tasks and environments, then evaluates which collected data actually helps the robot policy perform better, creating a feedback loop that guides future data collection toward more useful examples.

datatrainingapplications

Suppressing Misaligned Behaviors in Language Models, Durably

Nov 3, 2026

Luxi He, Pengcheng Jiang, Jifan Zhang et al.

Durable suppression of misaligned behaviors requires training against diverse misalignment states, not just the final bad behavior—PEARL does this by sampling intermediate states along cached trajectories during adversarial RL.

This paper studies how to durably suppress undesired behaviors in language models after they emerge. The authors create two test cases—models trained to cheat on multiple-choice questions or generate harmful code—then compare methods like standard fine-tuning and a new approach called PEARL that uses adversarial training with intermediate state sampling.

safetyalignmenttraining

Agnostic Learning by Imitating

Nov 3, 2026

Sílvia Casacuberta, Parikshit Gopalan, Varun Kanade et al.

Agnostic learning and multiaccuracy are computationally equivalent problems—you can efficiently convert solutions between them using adaptive queries, meaning advances in one area directly enable progress in the other.

This paper proves that two fundamental machine learning tasks are computationally equivalent: training a discriminative model to classify data as well as possible, and training a generative model whose outputs fool all classifiers in a given model class.

trainingevaluation

A Circuit for In-Context Learning

Nov 3, 2026

Andrew Gao, William Gao

In-context learning works through a mechanistic circuit of identifiable components (memory storage, key matching, gating) that you can trace causally from input to output, and this same circuit appears consistently across model sizes and architectures.

Researchers reverse-engineered how transformers perform in-context learning by identifying the specific circuit of attention heads and signals that enable few-shot classification.

reasoningtraining

CATF: A Manager-Worker Framework for Context-Aware Time-Series Forecasting

Nov 3, 2026

Yueqiao Chen, Dongyu Liu

Hard routing through a Manager model lets specialized Workers handle different data regimes automatically, improving forecasting accuracy while reducing inference costs compared to soft-routing ensemble approaches.

CATF is a forecasting framework that uses a Manager-Worker architecture to predict time-series values when multiple contextual factors (like weather or calendar) influence the data. A Transformer-based Manager intelligently routes each input to a specialized Worker model based on context, allowing workers to automatically specialize without manual data splitting.

architectureefficiencytraining
trainingefficiency

DecisionRank: On-Policy Distillation for Decision-Model Rerankers

Oct 30, 2026

Yanran Wu, Dian Yu, Chongyang Bai et al.

Decision models can match or exceed generative LLM ranking quality through on-policy distillation, while being orders of magnitude faster—enabling practical deployment for retrieval tasks.

DecisionRank trains a decision model to rank documents by predicting scores directly, avoiding slow autoregressive decoding. It uses a novel on-policy distillation method where a generative teacher LLM provides feedback on the student's predicted rankings, enabling a 1.7B model to outperform its 27B teacher while running 24× faster.

efficiencytrainingevaluation

Elucidating the Design Space of LM Data Attribution

Oct 29, 2026

Steven Yu, Sheng-Yu Wang, Kimia Hamidieh et al.

Data attribution for LMs works much better when you focus on the most informative tokens within documents using negative log-odds gradients instead of averaging attribution scores across entire documents—a simple fix that improves performance by 35-72 percentage points.

This paper diagnoses why data attribution methods for language models fail by creating a controlled benchmark with synthetic training data.

evaluationtrainingreasoning

ShallowDefend: Harnessing Shallow Safety Alignment for Multi-Turn Defender

Oct 29, 2026

Qiyao Ma, Xiaodong Liu, Weiwei Yang et al.

Safety vulnerabilities in multi-turn conversations stem from shallow alignment in early tokens; explicitly training models to commit to a parseable safety decision before generating responses—rather than just supervising the surface form—creates a causal control point that significantly impro...

This paper addresses why LLMs refuse harmful requests in single turns but fail across multi-turn conversations. The authors show safety alignment is 'shallow'—concentrated in early output tokens—and propose ShallowDefend, which makes models explicitly commit to a safety decision before responding.

safetyalignmenttraining

An ODE-based Sampler for Discrete Diffusion Models

Oct 29, 2026

Samson Gourevitch, Yazid Janati, Eric Moulines et al.

Discrete diffusion models can now use deterministic ODE-based sampling (like continuous diffusion models do), enabling faster inference and better distillation without sacrificing generation quality.

This paper develops a deterministic sampling method for discrete diffusion models by introducing exponential residual flows (ERFs), which use ODEs to guide token generation.

efficiencytrainingarchitecture

Difficulty Calibration Across Time for Membership Inference in Diffusion Models

Oct 29, 2026

Michael Joseph, Yedid Hoshen

Memorization in diffusion models can be reliably detected by comparing reconstruction losses across time (training or diffusion timesteps) rather than using absolute loss values, which conflate memorization with image difficulty.

This paper tackles membership inference attacks on diffusion models by separating true memorization from inherent image difficulty. The authors propose two calibration methods: Reference-Model Calibration (RMC) compares losses across training checkpoints, while Temporal Self-Calibration (TSC) uses a single model by comparing losses at different noise levels.

safetyevaluationtraining

Multi-View MoE: Letting Every Router Shape Expert Computation

Oct 29, 2026

Zelin Zhao, Tianchi Ma, Junyuan Shang et al.

Using multiple routers with different expert preferences improves expert selection in MoE models, and reusing routing signals to customize expert inputs boosts performance without increasing compute cost.

This paper improves sparse mixture-of-experts models by using multiple routers to select experts and modulate their computation. Instead of one router deciding which experts to use, multiple independently-trained routers vote on expert selection, and their preferences are reused to customize each expert's input representation.

architectureefficiencytraining

PIVOT: Progressive Intervention via Online Teacher Correction

Oct 29, 2026

Zhihao Yang, Boyuan Zheng, Puhui Tan et al.

Combining trajectory-level intervention (teacher corrections inserted into rollouts) with token-level supervision (learning from corrected segments) is more effective than either approach alone for training on hard reasoning tasks.

This paper addresses a key limitation in reinforcement learning from verifiable rewards (RLVR): when all attempted solutions fail, the model gets no learning signal. PIVOT combines two strategies—inserting teacher corrections into student attempts and using supervised learning on those corrections—to help models learn from harder problems where pure reward signals are too sparse.

trainingreasoningalignment

Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation

Oct 29, 2026

Yifu Luo, Chuang Ma, Zeyu Chen et al.

Token-level teacher selection during training outperforms fixed or response-level teacher assignment—different parts of a model's output benefit from different supervision signals.

This paper proposes MTA-OPSD, a method that assigns different teacher models to individual tokens during language model training. Unlike existing approaches that use a single fixed teacher or vary teachers only between responses, this method lets each token choose its best teacher through voting and KL divergence selection.

trainingefficiency

Gym-Anything for Robotics: Scaling physical simulations with digital agents

Oct 29, 2026

Pranjal Aggarwal, Sean Welleck

Using coding agents to generate simulations is a scalable way to create diverse, realistic robotics environments for training embodied AI, and combining multiple learning strategies (privileged information, self-learning, code-based policies) significantly improves performance on complex manipula...

This paper introduces Gym-Anything for Robotics, a framework where AI coding agents automatically create realistic robot simulations for training embodied AI models.

agentsapplicationstraining

Training Foundation Models with a Single Algorithm

Oct 29, 2026

Yuda Song, Alvin Zhang, Fahim Tajwar et al.

Foundation models don't need separate training algorithms for different stages—you can train everything with one likelihood-based method by treating RL objectives as fitting to model-generated distributions, which improves efficiency and capability coverage.

This paper shows that foundation model training stages (pretraining, fine-tuning, reinforcement learning) can be unified under a single likelihood-based algorithm. Instead of switching between different training methods, the researchers reformulate RL objectives as maximum-likelihood fitting, allowing all stages to use the same core algorithm.

trainingefficiencyreasoning

Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese

Oct 29, 2026

Pedro Carvalho Brom

Small language models trained on consumer hardware can match or exceed much larger models on specialized domains, but memorization and domain mismatch remain key challenges—this work provides practical benchmarks and a reusable toolkit for building domain-specific SLMs.

A tutorial on building a small language model (11-12M parameters) from scratch on a single consumer GPU for Brazilian Portuguese legal and general text.

trainingefficiencyevaluation

AI in the EGI e-infrastructure -- Past experiences and future strategies

Oct 29, 2026

Gergely Sipos, Sergio Andreozzi, Marco Rorro et al.

The AI landscape for research has shifted from building general models to adapting existing AI tools for specific scientific domains—EGI's strategy emphasizes infrastructure, skills, and trust as the foundation for this practical, domain-focused AI deployment.

EGI Federation shares 15+ years of experience deploying AI across European research infrastructure, from early digital twin projects to modern domain-specific applications. The paper outlines EGI's 2026-2030 strategy focusing on three pillars: AI-ready infrastructure connecting compute and data, workforce skills development, and trustworthy AI operations for high-risk research environments.

applicationstraining

Preserving Utility in Continual LLM Unlearning via Orthogonal Projections

Oct 29, 2026

Juan Belieni, Ana Carolina Erthal, Eliezer de Souza da Silva et al.

COP enables multiple sequential unlearning requests by mathematically preventing new updates from undoing previous deletions, storing only low-rank information per layer and working with existing unlearning methods.

This paper addresses the problem of repeatedly removing sensitive data from language models without retraining. When unlearning requests happen sequentially, standard methods degrade in performance.

safetytrainingefficiency

Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness

Oct 27, 2026

Igor Shilov, Xiaoxue Yang, Alexander Panfilov et al.

By learning which attention heads should focus on instructions vs. user input, you can defend against prompt injection attacks with minimal overhead—HoS reduces attack success rates from 97% to 10% while preserving the model's ability to use tools and answer questions correctly.

Heads of Security (HoS) is a defense against prompt injection attacks that works by training small attention gates for each transformer head to separate trusted instructions from untrusted user data. Unlike methods that require full model retraining, HoS adds minimal parameters while maintaining model utility and resisting both black-box and white-box attacks far better than existing defenses.

safetyarchitecturetraining

KPPax: preprocess differentiable kinetic ODE for machine learning in atmospheric chemistry

Oct 27, 2026

Wenqing Peng, Michael Boy, Zhi-Song Liu

By making atmospheric chemistry models differentiable, KPPax enables automatic computation of parameter sensitivities and hybrid physics-ML models without hand-coded adjoints—reducing emission estimation errors by 2.4–5× in demonstrated applications.

KPPax converts atmospheric chemistry models from KPP format into differentiable JAX code, enabling automatic differentiation and machine learning integration.

trainingefficiencyapplications

GEPA-Curator: Curated Learning for Self Improving Agents

Oct 27, 2026

Zack Yu, Nicholas Lee, Minjae Lee et al.

Not all agent failures are equally useful for learning—curating which tasks inform prompt optimization, and providing successful examples as context, significantly improves agent performance in complex multi-turn tasks.

GEPA-Curator improves how LLM agents learn from their mistakes by being selective about which failed tasks to learn from. Instead of learning from all failures, it filters out noisy or unreliable learning signals and retrieves successful past examples to guide prompt optimization, achieving up to 20.7% performance gains over existing methods.

trainingagentsreasoning

Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis

Oct 27, 2026

Andrew Szot, Samira Abnar, Omar Attia et al.

When training tool-use agents, prioritize diversity of tools and data sources over raw task quantity—this scales better and produces more capable agents than previous synthetic training approaches.

This paper studies how to train better LLM agents that use tools to complete multi-step tasks. The authors created SynthAPI, a system that generates synthetic training environments with controllable diversity across tools, tasks, and databases.

trainingdatascaling

MuonGS: Pretraining LLMs with Relaxed Weight-Norm Control

Oct 27, 2026

Athanasios Glentis, Dawei Li, Mingyi Hong

Relaxing weight-norm constraints to allow controlled growth (proportional to √t) improves optimizer efficiency and model convergence speed compared to strict norm-fixing approaches, with practical gains of ~15% on large-scale pretraining.

This paper analyzes how weight-norm control affects optimizer performance during language model pretraining and proposes MuonGS, a new optimizer variant. The key insight is that controlling how fast weight norms grow—rather than fixing them—allows better exploration during training.

trainingefficiency

What Improves During Pretraining? Tracking Knowledge Acquisition with Point-in-Time Models

Oct 27, 2026

Chang Su, Bin Wu, Emine Yilmaz et al.

Knowledge that appears inconsistently across web snapshots is learned faster during later pretraining stages and achieves better performance, suggesting that temporal variability in training data is a useful signal for understanding and predicting learning dynamics.

This paper investigates how language models learn different types of knowledge during pretraining by analyzing models trained on web data from different years.

trainingevaluationdata

EnvSched: Environment-Aware Scheduling for Rollout-Efficient Agent Reinforcement Learning

Oct 27, 2026

Sihan Yang, Weichen Bai, Ruichuan An et al.

By treating environments as first-class scheduling objects and predicting reward variance across task-environment pairs, EnvSched achieves 3x faster training than random selection—reaching in 40 steps what random sampling needs 120 steps to accomplish.

EnvSched is a scheduling system that makes reinforcement learning for language agents more efficient by intelligently selecting which tasks to train on within limited rollout budgets.

trainingefficiencyagents

Timestral

Oct 27, 2026

Adrien Petralia, Alexandre Gavaudan, Artjom Joosen et al.

Decoupling prediction tokens from causal context and pretraining on synthetic data enables time series models to forecast longer horizons more accurately—a practical approach for building better forecasting systems.

Timestral is a family of time series foundation models (5M–209M parameters) that achieves state-of-the-art probabilistic forecasting by combining three key techniques: Decoupled Horizon Forecasting (which trains longer predictions without masking context), native support for known-future covariates, and a two-stage pretraining pipeline on synthetic then real data.

scalingtrainingevaluation

REGER: Modeling inter-item Relations with Semantic IDs for Generative Recommendation

Oct 27, 2026

Yuxi Huang, Silin Zhou, Chenhao Wang et al.

By pretraining on semantic item relationships and using hierarchical preference decomposition, generative recommendation models can better capture both what items are similar and what users actually want to buy or interact with.

REGER improves generative recommendation systems by explicitly modeling relationships between items through semantic IDs. It pretrains models to learn item similarities, uses hierarchical attention to capture user history patterns, and applies preference distillation to better predict which items users will interact with next.

trainingreasoning

LCM-Promo: An LLM-Based Uplift Model for Promotion Allocation

Oct 27, 2026

Nesta Midavaine, Floris Fok, Paulo Mendes et al.

Fine-tuning LLMs on commerce transaction data can outperform traditional segmentation approaches for promotion targeting by learning individual customer price sensitivity directly from production logs.

LCM-Promo is an LLM fine-tuned on transaction data to decide which customers should receive promotions and at what discount level. Instead of treating promotion decisions as simple yes/no choices, it learns how different customers respond to different discount levels, then uses this to allocate promotions more efficiently.

applicationstrainingevaluation

Collective Dynamics of Self-Reinforcement: How Cross-Task Coupling Drives Policy Collapse in Unsupervised RLVR

Oct 27, 2026

Peize Ding

Policy collapse in self-reinforced language models isn't a training bug but a phase transition caused by cross-problem coupling—it becomes inevitable beyond a critical dataset size unless the model's shared representations are explicitly protected.

This paper explains why unsupervised reinforcement learning from a model's own majority votes causes catastrophic collapse—where the model gives identical answers to all problems despite initially improving.

trainingsafetyreasoning

Auto-Research Arena: Benchmarking and Advancing Auto-Research in Language Modeling

Oct 26, 2026

Xi Chen, Chongjian Ge, Yiran Xu et al.

Autonomous research systems need standardized benchmarks with clear success criteria; this work shows simple baselines often outperform sophisticated methods, suggesting current auto-research has significant room for improvement.

Auto-Research Arena is a benchmark for evaluating autonomous research systems that discover improvements in language modeling. It separates what counts as a valid result (admissibility) from what measures progress (targets), and tests seven different research methods across nine objectives like model efficiency and training speed.

evaluationtrainingagents

Preserve to Expand: Source-Aware Adaptation and Model Merging for Multilingual LLM-Based ASR

Oct 26, 2026

Yuchen Zhang, Haralambos Mouratidis, Ravi Shekhar

When adapting multilingual speech systems to new languages, simple fine-tuning causes catastrophic forgetting of old languages—but selective merging strategies like SSU-Wanda+TIES can preserve both old and new language performance simultaneously.

This paper addresses expanding multilingual speech recognition systems to new languages while preserving performance on existing languages, without replaying original training data. The researchers test different adaptation and merging strategies for a connector module in speech-based LLMs, finding that careful update merging balances learning new languages with retaining old ones.

trainingmultimodalefficiency

Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent

Oct 26, 2026

Benjamin Pikus, Daniel Saunders, Mircea Mironenco et al.

Pretrained language models can be efficiently adapted for robot control through simulation-based training alone, enabling strong real-world performance and compositional generalization without collecting real robot data.

ZAP is a robot control system that combines a pretrained vision-language model with simulation-based training to enable robots to understand tasks and execute them without real-world training data. The approach achieves 96.7% success on real-world manipulation tasks while being 3x faster and 17x cheaper than GPT-4 Astra, and generalizes to unseen objects, goals, and robot embodiments.

trainingapplicationsreasoning

Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models

Oct 26, 2026

Aidar Myrzakhan, Zhiqiang Shen

Diffusion LLMs can generate much faster by strategically choosing which tokens to commit to first—prioritizing tokens that unlock predictions for other positions, not just the easiest ones to predict.

This paper introduces dSpark, a method for speeding up diffusion language models by learning which tokens to reveal first during generation. Instead of just predicting easy tokens, the approach identifies tokens whose revelation helps other positions become easier to predict, reducing the total decoding rounds needed.

efficiencytrainingreasoning
trainingsafetyevaluation

Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning

Oct 23, 2026

Yijing Ke, Tianle Cai, Ruosong Wang

Training instability in LLM RL often stems from mismatched rounding locations between inference and training—fixing this in just a few critical computation paths can prevent collapse while keeping everything else unchanged.

This paper identifies how rounding differences between inference and training destabilize reinforcement learning in large language models, even when using identical weights.

trainingefficiency

Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation

Oct 23, 2026

Yixuan Xu, Hailin Liu, Linxiang Peng et al.

Knowledge distillation works better for semantic-ID recommenders when you match distributions either by reconstructing items from codes or by projecting item probabilities to individual code positions—both approaches beat standard hard-label training.

This paper tackles a key challenge in generative recommendation systems: how to transfer knowledge from traditional recommenders to newer semantic-ID models that predict items as sequences of discrete codes. The authors propose two knowledge-distillation methods that bridge the gap between item-level and code-level predictions, achieving significant improvements in recommendation quality.

trainingefficiency

Yalda-Embedding: Open Development of a Competitive Persian Retrieval Model

Oct 23, 2026

Yalda Taheri, Mohammad Hassan Heydari, Armon Rasooli

Language-specific retrieval models can match much larger models through targeted fine-tuning with distillation, and open-sourcing training data and procedures enables reproducible progress in underserved languages like Persian.

Yalda Embedding is a 596M-parameter Persian retrieval model fine-tuned from Qwen3-Embedding using over 1 million query-document pairs. It outperforms comparable 0.6B models on Persian retrieval tasks and approaches 4B model performance, with gains from knowledge distillation and contrastive learning. The authors release all code, weights, and training data to support Persian NLP research.

trainingmultimodalefficiency

Adaptive Frequency Threshold Byte Pair Encoding for Mixed Natural Language and Structured Symbolic Corpora

Oct 23, 2026

Pietro Di Nicco, Lucio La Cava, Andrea Tagarelli

Tokenization methods should adapt their merging criteria based on token characteristics—shorter tokens and domain-specific patterns need different frequency thresholds to get fair representation in the vocabulary.

This paper introduces AdaBPE, an improved tokenization method that adapts how subword units are created based on token length and type (general vs. domain-specific language).

trainingefficiency

Low-Bit N-Gram Embeddings for Language Models

Oct 22, 2026

Stephen Chung, Wenyu Du

You can compress n-gram embedding tables to 4-bit precision and save 75% of their memory with minimal performance loss—or use the freed memory to make the model better overall.

This paper shows that n-gram embedding tables in language models can be stored at low precision (4 bits) instead of full precision, reducing memory usage by ~75% while keeping 99% of the performance benefit. The authors test quantization methods on models from scratch and on large existing models like DeepSeek, finding that saved memory can be reinvested to improve overall performance.

efficiencytrainingarchitecture

Questioning Instrumental Convergence in RLVR: Checkpoint Evidence from OLMo-3

Oct 22, 2026

Barış Alparslan, Sandy Tanwisuth, Caleb Petterson

Instrumental convergence behavior in LLMs doesn't monotonically increase during RLVR training—it peaks mid-training then declines, challenging the assumption that RL automatically amplifies power-seeking tendencies.

This paper tracks how often a language model expresses instrumental convergence behaviors (like self-preservation or deception) across training checkpoints during reinforcement learning with verifiable rewards. Surprisingly, the rate rises early in training but then drops below the starting point, suggesting RLVR doesn't simply make models more power-seeking.

safetytrainingalignment

Action Learning for Scientific Simulations

Oct 22, 2026

Sarah Liaw, Yuanqi Du, Yilun Du

Action learning provides a unified, reusable objective for reasoning about dynamical systems from trajectory data—enabling path ranking, control, and inference without retraining for each new task.

This paper introduces LEAP, a method for learning stochastic actions from trajectory observations in dynamical systems. By anchoring unknown actions to a reference process and learning corrections through trajectory comparison, LEAP creates a differentiable objective that captures both local dynamics and long-term dependencies.

reasoningtrainingapplications

What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models

Oct 22, 2026

Jinya Sakurai, Xun Xu, Jaehong Yoon

For faster parallel token generation in diffusion LMs, a small learned policy (856K params) using confidence + hidden states can reliably decide which predictions to commit, improving throughput without sacrificing accuracy—and the policy transfers across different base models.

This paper studies how to decide which token predictions to commit to in parallel-decoding diffusion language models. The authors systematically analyze what makes a good unmasking policy—a learned model that decides when predictions are confident enough to finalize—by testing different supervision signals, inputs, and model sizes.

efficiencytrainingevaluation

When to Follow the Teacher: Advantage-Based On-Policy Distillation

Oct 22, 2026

Mahdi Sabbaghi, George Pappas, Hamed Hassani et al.

You can make distillation more efficient by learning which teacher actions actually improve student performance, rather than blindly copying everything the teacher does—and you can estimate this using only the data you already collect.

This paper proposes Advantage-Based On-Policy Distillation (AOPD), which improves how students learn from teachers by selectively following teacher actions based on whether they actually help the student succeed.

trainingreasoning

LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

Oct 21, 2026

Zhennan Wan, Jianfei Chen

You can train LLMs to handle longer contexts better using only self-generated data—no external corpora or teacher models needed—by having the model create its own reasoning tasks and learning from them.

LongZero enables LLMs to improve their long-context reasoning abilities without external data by having models generate their own training tasks. The method creates multi-hop reasoning and retrieval tasks by first synthesizing compact question-answer pairs with evidence, then expanding that evidence into long multi-document contexts, before training with reinforcement learning.

trainingreasoningefficiency

Asymmetric On-Policy Distillation Learns In-Context Exploration

Oct 21, 2026

Sriyash Poddar, Yanda Bao, Jacob Krantz et al.

You can teach agents efficient exploration by having a privileged teacher label actions conditioned on the student's real exploration history—no RL needed, just supervised learning with asymmetric information access.

ASTEROID is a student-teacher learning framework that teaches agents to explore efficiently under partial observability without reinforcement learning. A privileged teacher with full state information labels actions based on the student's actual exploration history, allowing the student to learn when and how to gather information.

trainingagentsefficiency

Differentially Private Sparse Reward Estimation with Preference Feedback

Oct 21, 2026

Meng Ding, Mingxi Lei, Jie Zhang et al.

By assuming sparse reward structure, you can learn from human preferences with differential privacy guarantees that depend only on the number of relevant features (s*), not the full model dimension (d)—making privacy practical for large language models.

This paper tackles privacy-preserving reward learning for aligning language models with human feedback. The key innovation is exploiting sparsity in reward models—assuming only a small subset of features matter—to achieve differential privacy guarantees that don't scale with model size.

alignmentsafetytraining

Evolving Agent Interfaces

Oct 20, 2026

Saaket Agashe, Yuzhe Yang, Gonzalo Gonzalez-Pumariega et al.

Interface design can be automated and improved through agent experience—evolving interfaces from task trajectories is a practical way to boost agent performance that's complementary to model scaling.

This paper introduces Agent S4, a system that automatically designs and improves computer interfaces for AI agents by learning from task experience. Rather than relying on hand-built interfaces, a meta-agent analyzes how task agents perform on training tasks and evolves better interfaces and usage strategies.

agentsarchitecturetraining

Text-to-Image Generative Modeling Without Language Pretraining

Oct 19, 2026

Keshigeyan Chandrasegaran, Yue Zhao, Armin W. Thomas et al.

You can build competitive text-to-image systems without relying on pretrained language models, which could make these systems cheaper and more accessible to build from scratch.

Halcyon shows that text-to-image models don't need large language pretraining to work well. By training from scratch on 100M image-text pairs, Halcyon achieves better results than existing systems while using far less compute—18× to 275× less than competitors. This challenges the assumption that internet-scale language models are essential for image generation.

trainingmultimodalefficiency

WeAct: Dynamic Weight Switching as Agent Actions

Oct 19, 2026

Rui Shi, Xunjian Yin, Jiacheng Sang et al.

Instead of permanently merging all skills into one model, let agents choose which skill module to activate at each step—this prevents forgetting old capabilities while enabling flexible, modular skill composition.

WeAct lets AI agents dynamically switch between specialized skill modules (LoRA adapters) during task execution, treating adapter selection as an action the agent can take. This solves catastrophic forgetting—where learning new skills degrades old ones—by keeping a frozen backbone and training separate adapters for different capabilities.

trainingagentsefficiency

Train An Imaginative Agent

Oct 19, 2026

Jiaxin Ge, Junyi Zhang, Tianning Chai et al.

Agents can learn to imagine and reason about action consequences before acting by combining world modeling with planning in a single model, dramatically improving performance in complex environments with irreversible actions.

This paper introduces Agentic Imagination (Agima), a training framework that teaches agents to simulate action consequences before executing them.

reasoningagentstraining

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Oct 19, 2026

Youran Dong, Junfeng Yang, Wei Yao et al.

Newton-based bilevel optimization can now handle non-unique solutions and singular Hessians by leveraging shared curvature between lower-level and multiplier updates, enabling faster convergence in practical ML tasks like hyperparameter tuning and data reweighting.

This paper develops Newton-based methods for bilevel optimization that work when lower-level solutions aren't unique and the Hessian is singular—situations where standard methods fail. The authors propose two new algorithms (Newton-based GALET and Newton-based BAMM) that exploit shared curvature structure and provide faster convergence guarantees.

training

Competing Learning in Hybrid Architectures

Oct 19, 2026

Xingyu Shen, Yingfa Chen, Ziqing Qiao et al.

In hybrid architectures mixing recurrence and attention, recurrence dominates learning of retrieval functions. Staged training that temporarily freezes recurrence allows attention to develop, significantly improving model performance.

This paper identifies a 'Competing Learning' problem in hybrid language models that combine recurrent and attention layers: recurrence learns retrieval functions faster, blocking attention from learning. The authors propose staged training—freezing recurrence updates mid-training—to let attention catch up, improving performance on language modeling and retrieval tasks.

trainingarchitectureefficiency
safetytrainingalignment

Beyond the Answer: Student-Conditioned Trace Construction for Reasoning Distillation

Oct 16, 2026

Shaoang Li, Yanhang Shi, Yufei Li et al.

Reasoning traces are more effective for training when customized to the student model's capabilities and selected based on how well they help recover answers, not just whether they're correct.

This paper presents ReDiST, a method for creating better reasoning traces to teach AI models. Instead of just showing correct answers, it constructs traces tailored to what a specific student model can learn—revising correct attempts and fixing wrong ones using evidence.

trainingreasoningevaluation

Learning Personalized Memory Gating from Delayed One-Sided Feedback

Oct 16, 2026

Kaixuan Ji, Zixiang Chen, Wenting Zhao et al.

Personalized memory gating—deciding which facts to remember for each user—can be learned efficiently by treating it as a bandit problem with delayed feedback, outperforming one-size-fits-all memory policies.

This paper tackles personalized memory management for AI agents by learning which facts to keep in a limited memory bank. The authors propose a method that learns user-specific patterns of what information gets reused later, combining offline learning from past data with online adaptation to individual users.

trainingagentsreasoning

CoDA: Co-Evolution of Data Engines and Action Policies

Oct 15, 2026

Tianjiao Yu, Zhilin Ou, Yifan Shen et al.

Co-evolving data generation and policy training through shared failure diagnosis and staged learning signals can dramatically improve robot learning efficiency and robustness to real-world variations.

CoDA is a framework where a robot learning system and a data generation engine improve together by sharing a memory of failures and solutions. A vision-language critic diagnoses why policies fail, and this feedback guides both the creation of challenging training environments and the design of better learning signals.

trainingagentsdata