ThinkLLM
ModelsCapabilitiesUse CasesBenchmarksPapersGlossary
ModelsCapabilitiesUse CasesBenchmarksPapersGlossary
AboutPrivacyTermsRSS

ThinkLLM

Spot an error in our data? Let us know.

Papers

Recent AI research papers with accessible summaries. Updated daily from arXiv, summarized for developers who don't read papers regularly.

2455 papers55 this month12 topics
AllEvaluation 43Agents 42Applications 39Efficiency 29Reasoning 29Training 27Safety 23Multimodal 18Architecture 9Alignment 8Data 4scaling 2

Nov 16 – Nov 22(14)

AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control

Nov 19, 2026

Yiyang Du, Yuxuan Kuang, Qitao Zhao et al.

Frontier multimodal agents show strong zero-shot robot control with good failure recovery, but action-level analysis reveals they still struggle with precise physical execution tasks—a critical insight for building more reliable robotic systems.

This paper evaluates how frontier AI agents (like GPT-4o) control robots by analyzing individual actions rather than just overall success rates. The authors introduce AgentRoboCritic, a framework where agents explain their intent before each action and a critic evaluates whether the action succeeded or failed.

agentsevaluationapplications

Bio-PM-Guided Synthetic IMU Generation

Nov 19, 2026

Deepali Singh, Deepesh Parpani, Aviral Gupta

Synthetic data generated in a pretrained encoder's embedding space can effectively augment real IMU data for activity recognition, especially for hard-to-distinguish activities, without requiring raw signal synthesis.

This paper generates synthetic IMU (motion sensor) data in the representation space of a pretrained biosignal encoder to improve human activity recognition across different people. Rather than creating raw sensor signals, the authors train decoders (autoregressive and diffusion-based) to generate token sequences in the encoder's space, then use these synthetic tokens to augment training data.

Nov 9 – Nov 15(17)

Training Reasoning Models in Simulation to Natively Use Real Robots

Nov 13, 2026

Jiaming Tang, Zhuoyang Zhang, Junxian Guo et al.

Open-weight reasoning models can learn effective robot control through simulation-based reinforcement learning, achieving real-world performance comparable to frontier models without demonstrations or task-specific fine-tuning.

Researchers trained an open-weight 27B reasoning model to control real robots by learning entirely in simulation through reinforcement learning, without needing human demonstrations. The model observes the environment, reasons through its actions, and generates robot commands—similar to how AI agents use computers.

trainingagentsapplications

StreamEmbed: Real-Time Indexing from Pixels for Screenshot Retrieval

Nov 13, 2026

Dayoon Ko, Yichuan Wang, Soyeong Jeong et al.

By reusing computations across incrementally-changing screenshots, you can index screen history 3.7x faster in real-time without sacrificing visual understanding—making local visual assistants practical.

StreamEmbed enables real-time visual indexing of screenshots by exploiting pixel redundancy across frames. Instead of recomputing embeddings from scratch, it uses spatial token pruning to skip unchanged image blocks and temporal key-value reuse to cache computations across screenshots, achieving 3.7x faster encoding than standard visual encoders while maintaining accuracy.

Nov 2 – Nov 8(24)

CallCleaner: Lightweight Bioacoustic Denoising from an Audio Foundation Model

Nov 8, 2026

Michael Xie, Gašper Beguš, Morris Alper

By distilling a general-purpose audio foundation model, researchers created a practical denoiser for animal recordings that's both faster and more effective than specialized bioacoustic methods, showing that foundation models can be efficiently adapted for domain-specific audio tasks.

CallCleaner is a lightweight audio denoising tool for animal recordings that distills a large foundation model into a 19M-parameter neural network. It removes background noise from field recordings of animal vocalizations without requiring voice detection, runs 90x faster than real-time on a laptop CPU, and outperforms existing bioacoustic denoisers while maintaining call quality.

efficiencyapplications

GOPR-CAD: Graph-of-Operations Progressive Reasoning for CAD Generation

Nov 8, 2026

Yitian Pang, Yifan Li, Xiaowei Yue et al.

Decomposing CAD generation into discrete operations that an LLM selects sequentially, rather than generating full code at once, dramatically improves structural correctness (82.7% success rate) and works with any language model without retraining.

This paper presents GOPR-CAD, a system that generates CAD models from text by having an AI agent progressively build a graph of operations rather than directly writing code.

Oct 26 – Nov 1(21)

RoboArchitect: Automating the Design of Robotic Agents with Persistent Memory

Oct 30, 2026

Shengguang Wu, Suning Huang, Cherie Ho et al.

You can dramatically improve robot performance by automating harness design—the control logic wrapper around a policy—rather than retraining the policy itself, achieving specialist-level performance from general-purpose models.

RoboArchitect automates the design of control harnesses around frozen robot policies using a coding model that observes failures and iteratively revises the harness code. Without updating model weights, it improves success rates 4.5x on unseen scenes and 3.2x on unseen tasks by automatically discovering strategies like obstacle avoidance and persistent memory for learned behaviors.

agentsreasoningapplications

From extreme weather to agricultural consequences with agentic AI

Oct 29, 2026

Minghan Yu, Mengyu Dai, Yixin Wen et al.

Agentic AI can bridge the gap between weather forecasting and agricultural impact prediction by combining multiple data sources through causal reasoning, enabling faster disaster response for farmers and policymakers.

WxCrop is an AI agent that predicts crop losses from extreme weather by combining weather forecasts with agricultural data. It uses physically interpretable weather indices and causal analysis to estimate which crops will be damaged and by how much, validated on hurricane, freeze, and drought impacts in Florida with results matching insurance records and production data.

Oct 19 – Oct 25(14)

SceneBloom: Agentic Indoor Environment Generation with Realistic and Diverse Layouts for Embodied Learning

Oct 23, 2026

Yuhan Liu, Lianhui Qin, Shenji Wan

By combining diffusion-based visual layout generation with geometric constraint-solving, SceneBloom creates training environments so realistic that policies trained on them outperform those trained on real scans—solving a key bottleneck in embodied AI.

SceneBloom is an AI system that generates realistic 3D indoor environments for training embodied agents (robots/AI that interact with physical spaces). It combines visual layout generation with geometric reasoning to create physically valid room layouts with furniture arrangements, achieving better realism and stability than existing methods.

agentsapplicationsmultimodal

Measure Before You Train: Typed Decision Models vs. a Classical Baseline vs. a Generative LLM for Routing a Personal Multilingual Inbox

Oct 23, 2026

Federico Lucca

Before deploying AI systems, measure ground truth carefully and optimize for the actual business metric, not just accuracy—a perfectly accurate classifier can fail catastrophically if it ignores rare but important cases.

This paper evaluates email routing systems for a personal multilingual inbox using rigorous measurement before training.

Oct 12 – Oct 18(10)

AI-Driven Reproducibility and Analytical Robustness in Scientific Discovery

Oct 16, 2026

Shicheng Liu, Krista Opsahl-Ong, Andrew Drozdov et al.

LLM agents can automate reproducibility checks and robustness testing in scientific research, flagging analytical choices that might invalidate findings—reducing manual review burden while improving research quality.

This paper develops AI agents to automatically reproduce scientific findings and test their robustness. Using medical studies from a cancer registry, the system successfully reproduces 77% of claims and identifies analytical choices that could affect conclusions, helping experts focus on potentially problematic research decisions.

agentsevaluationapplications

Self-Building Labs

Oct 16, 2026

Jiaru Bai, Yuchi Zhao, Han Hao et al.

Instead of manually rebuilding labs for new experiments, AI can now autonomously design lab infrastructure and workflows from high-level scientific goals, making experimental automation dramatically more flexible and scalable.

Self-building laboratories (SBLs) are AI systems that automatically design and configure their own lab setups to achieve scientific goals, starting from scratch or existing equipment.

agents
dataefficiencyapplications

SoK: Assessing the Evolution of LLM Capability in Software Vulnerability Research

Nov 19, 2026

Amine Lbath, Manan Suri, Aurelien Delaitre et al.

LLM-based vulnerability discovery is already autonomous and real-world impactful: vulnerability disclosures doubled in 2026 with more critical-severity bugs found, but the security community hasn't yet adapted response timelines to match this acceleration.

This paper systematizes evidence on how LLMs are discovering, validating, and patching software vulnerabilities. The authors create an autonomy framework to rate LLM capabilities, review published claims, and analyze real-world vulnerability data from 2026 onwards.

safetyevaluationapplications

Verify the Verifier: Specification-Grounded LLM Cross-Checking for Bugs That Escape UVM Scoreboards

Nov 19, 2026

Ana E. Sanchez Villalobos, Michael D. Quinn, Aakash Tyagi et al.

LLMs can augment hardware verification by cross-checking test traces against specifications to find bugs that traditional scoreboards miss, catching 41 real scoreboard escapes in OpenTitan IP blocks.

This paper presents a framework using LLMs to find bugs that slip through hardware verification testbenches. Traditional verification checks if a design matches a reference model, but misses bugs that violate the actual specification. The framework analyzes test execution traces against the specification to catch these 'scoreboard escapes'—bugs that pass all tests but break the spec.

evaluationapplications

GEOHunter: Mapping a Generative Engine Optimization Network in the Wild

Nov 19, 2026

Sean Tang, Peiran Wang, Ying Li et al.

Generative engines are vulnerable to coordinated manipulation through user-generated platforms, where paid promotional content can systematically influence AI-synthesized answers at scale—requiring defenses beyond content filtering.

This paper investigates paid content networks that manipulate generative AI engines by publishing promotional articles on user-generated platforms.

safetyevaluationapplications

Cost-Aware Agentic Feature Optimization System for Fleet-Scale Ranking Models

Nov 19, 2026

Chang Yang, Yao Xuan, Wendan Yan et al.

LLM agents can effectively optimize complex multi-model systems by proposing hypotheses and using deterministic tools to verify them, achieving real production gains while respecting hard resource constraints.

A system that uses LLM agents to automatically optimize which features ranking models use, balancing performance improvements against computational costs. The system proposes feature combinations, ranks them by importance, and iteratively refines selections under budget constraints—improving production metrics by 0.20% while reducing pipeline costs.

agentsapplicationsefficiency

Backtracking Agent Applications Security Incidents

Nov 19, 2026

Yicheng Cai, Taegyu Kim, Peng Liu

Agent applications need specialized security incident response tools—Origin shows how to automatically map between agent-level behavior and code-level execution to quickly find attack entry points.

Origin is a security tool that helps analysts trace the source of attacks in AI agent applications by bridging the gap between high-level agent behavior logs and low-level code execution. When a security incident occurs, analysts use Origin to query trajectory logs and identify where the compromise started, without manually inspecting every line of code.

safetyagentsapplications

Agentic Harness Engineering Beta: Requirement-Centered Evolution of Agent Harnesses

Nov 18, 2026

Jiahang Lin, Jiahao Chen, Yiding Wang et al.

Requirement-driven harness evolution—where you identify user needs, implement generalizable solutions, and validate against those needs—produces more capable and practical AI agents than benchmark-focused optimization alone.

This paper presents AHE-Beta, a method for improving AI agent harnesses (the tools and interfaces agents use) by centering development around actual user requirements.

agentsevaluationapplications

Who are you? The Personal Imitation Game

Nov 18, 2026

Muhammad Rizky Wellyanto, Hajara-Yasmin Isa, Kexin Xue et al.

LLMs can now convincingly mimic specific individuals' conversational patterns and personality traits, achieving near-human-level impersonation in controlled settings—a capability with significant implications for identity, authentication, and privacy.

This paper proposes the Personal Imitation Game, a harder version of Turing's test that asks if an AI can convincingly impersonate a specific person rather than a generic human. Researchers built an LLM system trained on one author's 10 years of personal messages and tested it against human interrogators.

evaluationapplications

Ground, Iterate, Trace: Decomposing AI Research Ideation

Nov 18, 2026

Samson Yu, Qiran Zou, Jing Yu Lim et al.

Combining literature grounding with explicit, traceable reasoning operators improves AI-generated research ideas, but the quality gains come more from iterative refinement than from the specific operators themselves—the real value is in inspectability and source attribution.

This paper presents CAR, an AI framework that generates research ideas by grounding them in literature, iteratively developing concepts, and explicitly tracking the reasoning process. CAR retrieves relevant papers, builds a knowledge graph, and uses four conceptual operators (bisociation, analogy, exploration, elaboration) to transform ideas while recording each step and its sources.

agentsreasoningapplications

Local AI Agents in Power Electronics Simulation: Deployment, Measurements and Costs

Nov 18, 2026

Yuzhuo Li, Pasan Gunawardena, Yunwei Li

Local AI agent deployment for power electronics simulation trades off data privacy and control against measurable costs in power, energy, and time—organizations need quantitative benchmarks to decide whether local or cloud deployment makes sense for their specific tasks.

This paper evaluates locally-deployed AI agents for power electronics simulation tasks, measuring their power consumption, energy use, token generation, and execution time. Unlike cloud-based approaches, local deployment keeps sensitive designs and data on-site but creates computational overhead.

applicationsefficiencyevaluation

Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws

Nov 18, 2026

Zhiyu Liu, Xinyu Yang

LLMs can reliably detect major flaws in black-box clinical models through query access alone, but their reports of problems and new discoveries require verification against the actual model and real data—they generate many false positives.

Researchers tested whether GPT-4 Astra can audit clinical prediction models by querying them and reading predictions. They trained models on stroke data, planted 24 deliberate flaws, and had the LLM find them blind.

evaluationsafetyapplications

Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling

Nov 18, 2026

Eugene Agichtein, Charles L. A. Clarke, Kaustubh Dhole et al.

You can replace expensive LLM judges with one-time program generation that's faster, cheaper, and fully inspectable—achieving comparable accuracy to frontier models while costing just $0.17-0.49 per topic instead of per-document LLM calls.

This paper proposes ExecJudges, a method that uses LLMs to write readable programs instead of making individual relevance judgments. These programs use lexical matching and NLP tools to automatically label documents, eliminating expensive per-document LLM calls while remaining interpretable and reproducible.

evaluationefficiencyapplications

Vibe-Trading: Towards Agent-Native Trading

Nov 18, 2026

Haozhe Wu, Zhonghang Li, Chao Huang

Financial trading agents need special architecture to maintain traceability and authority control—this framework shows how to keep research results tied to their data snapshots, timing, and execution context so decisions can be verified and reproduced.

Vibe-Trading is a framework for building AI agents that can trade financial markets reliably. Unlike general-purpose agents, it solves the problem of keeping trading decisions connected to their source data, timing, and execution context as markets change.

agentsapplicationsreasoning
efficiencymultimodalapplications

MeReader: An Offline, Privacy-Preserving, Progress-Aware AI Assistant for Narrative eBook Reading

Nov 13, 2026

Mohammed Efaz, Udo Bub, Itilekha Podder

You can build privacy-preserving, spoiler-free reading companions using small local language models (2-4B parameters) that match larger models' performance while running entirely offline on consumer hardware.

MeReader is an offline AI assistant for reading ebooks that keeps your reading private and prevents spoilers by only accessing content up to your current position in the book. It uses a hybrid search system combining multiple retrieval methods and works efficiently on consumer devices, showing that smaller local models can match larger cloud-based systems in comprehension quality.

applicationsefficiencysafety

Agentic induction of common data elements and information extraction from sarcoma case report abstracts

Nov 13, 2026

Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah et al.

LLM agents can efficiently draft data schemas from unstructured medical text and extract structured information at scale, but human expert validation remains essential for ensuring accuracy and clinical relevance.

Researchers used AI agents to automatically extract and organize medical data elements from thousands of cancer research abstracts. Starting with nearly 73,000 candidate data fields, they refined them to 53 validated fields and successfully extracted information from 79% of abstracts, with human reviewers confirming 83% were relevant.

applicationsdata

Agentic AI for Discovering Fault-Tolerant Quantum Circuits

Nov 13, 2026

Aritra Lahiri, Florian Marquardt

Agentic AI can automate the manual design of quantum error-correcting circuits by combining LLM reasoning with evolutionary search and formal verification, potentially accelerating quantum computing development.

Researchers use AI agents to automatically design fault-tolerant quantum circuits—a critical bottleneck in scaling quantum computers. An LLM guided by a circuit verifier evolves programs that generate quantum error-correcting circuits for tasks like state preparation and logical gates, verified against all possible fault scenarios.

agentsreasoningapplications

ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

Nov 13, 2026

Dhruv Dixit, Khushi Patel, Congcong Xu et al.

Small models can match larger ones on math reasoning by learning to produce executable, tool-grounded action traces instead of free-form explanations, combined with reinforcement learning that rewards both correctness and process quality.

ARCHER trains small language models to solve math problems by learning to produce executable reasoning traces—sequences of reasoning steps, calculator calls, and verification actions—rather than just explaining their work.

trainingreasoningapplications

Metal Sandbox: Isolated Execution and Command Scheduling for GPU Coding Agents on Apple Silicon

Nov 12, 2026

Haozhe Wang, Lina Chen, Fanqi Wan et al.

When building AI agents that generate GPU code, isolating execution environments and intelligently scheduling commands across multiple devices can nearly double throughput and significantly reduce idle time.

Metal Sandbox is infrastructure for efficiently training and evaluating AI agents that generate GPU code for Apple Silicon. It isolates agent workspaces, schedules GPU commands across multiple Macs, and lets waiting commands migrate to available machines—achieving 1.68–2.05× throughput gains and reducing tool waiting time by 18.8%.

efficiencyagentsapplications

Programming Quantum Circuit Parameters with Large Language Models

Nov 12, 2026

Tiago de Souza Farias, Caio Moraes Porto, Matheus da Silva Fonseca et al.

LLMs can generate quantum circuit parameters effectively only when problems have identifiable structure; for general cases, they underperform conventional optimization, and their energy predictions need independent verification.

This paper investigates whether large language models can generate parameters for quantum circuits without traditional optimization.

evaluationapplications

Scalable Conversational Assessment: Chat-Based Exams with a Two-Agent LLM Pipeline

Nov 12, 2026

Geoffrey Challen, Craig Zilles

A two-agent LLM pipeline can conduct scalable conversational assessments that mimic oral exams by separating the interviewer from the evaluator, reducing cheating risks while maintaining the ability to ask follow-up questions and assess reasoning.

This paper presents an LLM-based system that conducts interactive text-based conversations with students to assess their reasoning and argumentation skills, mimicking oral exams at scale. It uses a two-agent architecture where an interviewer asks questions while a hidden evaluator tracks progress and prevents answer leakage, tested with simulated adversarial students and deployed in a real course.

applicationsevaluationagents

Cortex-FL: Agent-Mediated Collaboration for Federated Learning

Nov 12, 2026

Rui Sun, Jinfeng Peng, Zehua Cheng et al.

This system makes federated learning practical for non-experts by automating the hard parts (finding partners, negotiating terms, managing incentives) while preserving data privacy and organizational autonomy through smart contracts.

Cortex-FL is a federated learning system where AI agents help organizations collaborate on model training without sharing raw data. Users describe what model they want in plain English, and agents automatically find partners, negotiate agreements, and coordinate training—while keeping each organization in control of their own data and finances.

agentstrainingapplications

Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation

Nov 12, 2026

Tanzeel Abbas, Tehseen Zia, David Windridge

RAG systems frequently misattribute claims to retrieved sources when they actually come from parametric memory, and current citation verification methods cannot reliably detect this—a critical issue for trustworthy AI systems.

This paper reveals that retrieval-augmented generation (RAG) systems often cite sources they didn't actually use, instead relying on knowledge baked into the model itself. By replacing facts in retrieved passages with false versions and re-querying the model, researchers found that 1.2–15.7% of cited claims come from the model's internal memory rather than the provided context.

evaluationsafetyapplications

Unibuc - NLP at "Voight-Kampff" Generative AI Detection PAN 2025

Nov 12, 2026

Teodor Marchitan, Claudiu Creanga, Liviu P. Dinu

Ensemble methods using embeddings from LLMs with classical ML models can match or exceed fine-tuned LLM performance for AI-generated text detection, suggesting embeddings alone capture sufficient signal for this task.

This paper presents methods for detecting AI-generated text and classifying human-AI collaborative writing. The team experimented with LLM embeddings combined with classical machine learning models (achieving 99.4% F1) and transformer-based feature extraction for two related detection tasks at PAN 2025.

evaluationapplications

Pyrite: Building and Benchmarking Protein-Ligand Pose Ranking Agents

Nov 10, 2026

Megha Srivastava, Michael H. Lim, Allison M. Keys et al.

LLMs can discover useful features from molecular data better than direct reasoning, and this feature-discovery approach beats expert chemists at ranking binding poses—a critical step in drug design.

Pyrite is a benchmark for ranking protein-ligand binding poses in drug discovery, where LLM-based agents outperform human experts. The paper introduces two AI approaches: one that reasons about poses directly, and another that generates code to discover predictive features, achieving up to 74% accuracy on 1,500 pose pairs.

applicationsagentsreasoning

Emergent Biological Capabilities in a Foundation Model for Molecular Interactions

Nov 10, 2026

Adi Mashiach

Foundation models can develop unexpected biological reasoning abilities through scale alone, and these emergent capabilities don't follow smooth scaling laws—they appear suddenly at larger model sizes, suggesting qualitative jumps in what AI can do with molecular data.

Bonbon is a foundation model trained on 3.2 trillion tokens of protein-ligand sequences that unexpectedly develops three distinct capabilities for predicting molecular interactions—mechanism of action, binding site location, and bond type—without explicit training for these tasks.

scalingmultimodalapplications

ReTrace: Distilling Tool-Using Agents into Planner-Free Workflows

Nov 10, 2026

Kaiyuan Hou, Yang Ping Kuo, Indranil Bhattacharya et al.

You can replace expensive LLM calls in repetitive business processes with rule-based code by learning decision logic from agent execution traces—achieving near-agent performance at a fraction of the latency and cost.

ReTrace converts LLM agent workflows into fast, cost-efficient code-based processes by extracting decision rules from agent reasoning logs. It analyzes why agents took actions, distills these explanations into compact rule documents, and generates executable workflows that run 130x faster at 24% lower cost while maintaining 93.8% success on production tasks.

agentsefficiencyapplications

Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

Nov 10, 2026

Evelyn Y. Garland, Carola F. Berger

Using LLMs as judges with careful prompt engineering and ROC-guided model selection can deliver competitive translation quality evaluation at a fraction of typical inference costs.

Pterodactyl is a cost-efficient system for evaluating machine translation quality at the segment level. It uses an LLM-as-a-judge approach with optimized prompting to predict quality scores and identify error-free translations, designed for practical deployment in translation workflows.

evaluationapplications

C$^2$-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference

Nov 10, 2026

Guorui Lu, Qinyu Chen

Cropping and compression are complementary strategies for reducing VLM inference cost—use cropping for detail-focused questions and compression for context-heavy ones, with a learned safety mechanism to protect accuracy.

This paper proposes C²-Routing, a method that adaptively chooses between cropping (for local details) and compression (for context) to reduce computational costs in vision-language models for VQA tasks. It includes a safety gate to prevent accuracy loss and achieves 36.53% FLOP reduction with minimal performance drop.

efficiencymultimodalapplications
agentsreasoningapplications

RealEmbedBench: Benchmarking LLMs on Real-World Application-Layer Embedded Software Tasks

Nov 8, 2026

Yujun Huang, Yuwei Zhang, Yi Zhou et al.

LLMs show uneven performance on embedded software tasks: they need local code context to generate code effectively, can model state machines but fail at implementing them, and struggle with certain OS-to-OS translations, suggesting developers should choose models carefully and provide surrounding...

RealEmbedBench is a benchmark with 5,142 real-world embedded software tasks testing LLMs on code generation, state machine modeling, and cross-OS translation.

evaluationapplications

SWHForest: A Structural Curation and Search Layer for Large-Scale Code Corpora

Nov 8, 2026

Federico Pennino, Andrea Gurioli, Maurizio Gabbrielli et al.

For code model pre-training, you can now query structural properties (AST patterns, definitions, etc.) across massive corpora in seconds without downloading everything—enabling faster, more targeted dataset curation.

SWHForest is a framework that creates a searchable structural index of code files without storing source code, letting researchers efficiently filter and curate large code corpora before downloading. Instead of downloading millions of files to find what they need, users can query the index to identify matching files first, reducing downloads by 46-72%.

datatrainingapplications

Flex-DDCM: Zero-Shot Task-Oriented Image Compression via Diffusion Trajectory Guidance

Nov 8, 2026

Yixin Gao, Runsen Feng, Xin Li et al.

You can compress images for specific AI tasks (like object detection) without retraining by steering a pretrained diffusion model's denoising process with task guidance signals—enabling one codec to serve multiple purposes.

This paper presents Flex-DDCM, a zero-shot image compression method that uses diffusion models to compress images while optimizing for specific downstream tasks. By guiding the diffusion denoising process with task-specific signals, it achieves flexible trade-offs between image quality and task performance without requiring task-specific training.

efficiencymultimodalapplications

On Agentic Manipulation Policies

Nov 6, 2026

Chen Chen, Omid Bagherieh, Naveen Nikamanth Appiah Balaji et al.

Generalist AI agents controlling robots directly outperform specialist manipulation policies by 15-20x in failure rates, and can learn from experience with memory—suggesting agentic approaches are more robust and general than task-specific models.

This paper shows that generalist AI models can control robot manipulators more reliably than specialist policies by using agentic manipulation policies (AMPs)—where an AI agent directly commands the robot and adapts actions in real-time. AMP1, their reference system, achieves 85.6% zero-shot success on RoboLab-120 (vs.

agentsapplicationsreasoning

GRAC: Graph-Based Retrieval-Augmented Captioning for Surgical Scene Description

Nov 6, 2026

Rochak Dhakal, Chen Zhao, Soup Cobb et al.

Combining structured scene graphs with retrieval-augmented generation substantially improves surgical scene captioning accuracy and factual correctness, suggesting that domain-specific structured representations can enhance LLM-based medical documentation.

GRAC is a system that automatically generates descriptions of surgical scenes by combining graph representations of surgical elements (instruments, actions, anatomy) with retrieval-augmented generation. It retrieves similar past surgical scenes and uses them to guide caption generation, achieving significantly better descriptions than existing methods on surgical video benchmarks.

multimodalapplications

Computer-Use Agents Are Embodied Generalists

Nov 6, 2026

Pranjal Aggarwal, Sean Welleck

General-purpose computer-use agents can solve diverse embodied tasks without specialization, pointing toward a unified approach to embodied AI rather than task-specific models.

This paper shows that computer-use agents—AI systems trained to interact with software via screenshots and mouse/keyboard actions—can perform physical embodied tasks by rendering the environment on a screen.

agentsapplicationsmultimodal

Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization

Nov 6, 2026

Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj et al.

You can assess LLM-generated code summary quality by measuring inter-model agreement and using an LLM-as-judge, eliminating the need for expensive human reference summaries.

This paper proposes a method to evaluate whether AI-generated code summaries are correct without needing human reference summaries. It uses multiple AI models to generate summaries, has another AI model compare them pairwise, and combines these judgments with a calibrated rule that rewards agreement and penalizes only confirmed disagreements.

evaluationapplications

Can Agentic Systems Close the Gap in Radiology with Production AI Tools?

Nov 6, 2026

Aubin Charley, Julien Khlaut, Charles Corbière et al.

Agentic systems can bridge the gap between narrow specialist radiology AI and broad generalist models, but only if they have access to the right tools and can effectively decide which tools to use—tool availability and orchestration strategy are equally important constraints.

This paper studies how agentic systems—AI orchestrators that coordinate multiple specialized tools—can improve radiology reporting. Researchers built a framework where an orchestrator calls modular radiology AI tools to generate structured reports, then tested it against FDA-cleared devices.

agentsapplicationsevaluation

Discovering formally verified quantum programs with AI

Nov 6, 2026

Sergey Shirobokov, Ben Jaderberg, Katerina Hristova et al.

LLM agents can discover novel quantum algorithms faster than humans by working within formal verification systems, enabling optimization of algorithms that classical computers can't fully test.

Researchers used AI agents to automatically discover and formally verify quantum algorithms in the Lean theorem prover. The system achieved significant speedups on quantum computing tasks—up to 290x on arithmetic and 11x on fusion simulations—by having LLM agents synthesize programs against formal specifications, collaborate through shared wikis, and verify correctness mathematically.

reasoningapplicationsagents

User Embeddings are Superpositions of Interpretable Behavioral Modes

Nov 5, 2026

Jonathan Kozaczuk, Zheng Dong, Noyan Evirgen et al.

User embeddings contain interpretable behavioral patterns that can be extracted post-hoc using sparse autoencoders and automatically labeled—making opaque recommendation systems more transparent without changing existing infrastructure.

User embeddings in recommendation systems are dense and hard to interpret. This paper shows that embeddings can be decomposed into sparse, interpretable behavioral modes using sparse autoencoders. The recovered modes can be automatically labeled in natural language, enabling zero-shot audience retrieval without retraining the original system.

applicationsefficiency

XStack: A Benchmark for ML Repo Migration

Nov 5, 2026

Linrong Cai, Taiming Lu, Haozhe Chen et al.

Migrating ML code between frameworks and hardware is hard enough that it serves as a meaningful benchmark for AI agents, and today's best models still fail on a quarter of real-world migration tasks.

XStack is a benchmark that tests AI agents' ability to migrate machine learning repositories from PyTorch (NVIDIA GPUs) to JAX (Google TPUs). Researchers created a grading system based on published results to evaluate whether migrated code works correctly.

evaluationapplicationsdata

ReproBridge: Building Reproducibility into ML Research Artifacts

Nov 5, 2026

Yubo Wang, Rui Meng, Jaehyun Nam et al.

Packaging ML experiments with clear procedures and environment details—not just raw code—dramatically improves AI agents' ability to reproduce results reliably and cheaply, regardless of the agent's base capability.

ReproBridge helps AI agents reproduce machine learning experiments by creating standardized, refined artifacts that bundle code, environment setup, and execution guides. Testing on 1,029 tasks shows refined artifacts boost reproduction success from 76% to 93% and cut API costs by 20%, while narrowing performance gaps between different AI models.

evaluationtrainingapplications

MedGraph: Maintaining Patient State with Graph Memory for Long-Term Medical Dialogue

Nov 5, 2026

Donghua Zhang, Zizheng Wang, Ze Dong et al.

For medical AI systems handling long-term patient care, structured graph memory that tracks temporal changes and explicitly maintains safety facts outperforms generic retrieval methods by a significant margin.

MedGraph is a graph-based memory system for medical chatbots that track patients across multiple visits. It solves three key problems: outdated information, mixing up different patients' facts, and missing safety-critical details.

applicationsdatasafety

The Power of Meta Access in the Software of the Future --- Demonstrated on LLM Chat UIs and Chat AI

Nov 5, 2026

Julian Kunkel, Ali Doosthosseini

By exposing application state as structured data to AI assistants, you can transform chat interfaces from passive conversation windows into active workspace controllers, enabling assistants to help users navigate, organize, and operate complex tools—but only with proper permission boundaries an...

This paper introduces 'meta access'—giving AI assistants a structured, permissioned view of the application interface itself—turning chat from a single-conversation window into a control layer for entire workspaces.

agentsapplicationssafety

Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks

Nov 5, 2026

E. Kelly Buchanan, Crystal Zhou, Scott W. Linderman et al.

Benchmarks require ongoing maintenance as models improve—not just fixing bugs in tasks, but also revalidating resource constraints and evaluation assumptions that can materially change what capabilities are actually being measured.

This paper addresses how benchmarks for coding agents become outdated as models improve and infrastructure changes. The authors introduce a continuous validation workflow for maintaining benchmarks, apply it to Terminal-Bench 2.0, and find issues in ~30% of tasks.

evaluationagentsapplications

MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation

Nov 5, 2026

Sateesh Kumar, Arnav Balaji, Arpit Bahety et al.

By reconstructing 3D scenes and manipulated objects separately, then re-rendering demonstrations from novel robot base poses, MOMASPLAT makes visuomotor policies robust to viewpoint changes—doubling success rates on unseen positions without collecting more real-world data.

MOMASPLAT augments robot manipulation demonstrations by synthesizing them from unseen camera viewpoints using 3D reconstruction and novel-view synthesis. This solves a key problem in mobile manipulation: robots rarely execute tasks from the exact same position as during training, causing policy failures.

dataapplicationsmultimodal

Dull, Dirty, Dangerous? Working Conditions and the Limits of Occupational AI Exposure

Nov 4, 2026

Siyi Yu

AI exposure doesn't clearly target the worst jobs; working conditions explain only a small part of which occupations face automation, suggesting AI adoption is driven by other factors like skill requirements and economic value rather than worker relief.

This paper examines whether AI automation targets jobs that are dull, dirty, or dangerous—the traditional justification for automation. Using data from 754 US occupations, the author finds that higher AI exposure is weakly associated with worse working conditions, but this relationship largely disappears after accounting for education, wages, and physical demands.

evaluationapplications

From Autonomous Agents to Cognitive Symbionts: Toward AI-Augmented Human Intelligence

Nov 3, 2026

Hao Fei

The future of AI isn't just autonomous machines doing tasks alone—it's intimate, context-aware systems that live alongside people, learn their minds over years, and amplify human capability in real-time while respecting privacy and autonomy.

This paper proposes AI-augmented human intelligence (AHI) as a new paradigm where AI systems act as 'cognitive symbionts'—lifelong, embedded companions that augment what people can think and do by bridging the gap between their potential and working memory limits.

agentsapplicationsalignment

PolicyIK: High-Precision Inverse Kinematics for Robot Arms with Reinforcement Learning

Nov 3, 2026

Yuhan Liu, Xinqi Liu, Zixuan Dong et al.

Reinforcement learning can achieve higher-precision inverse kinematics than classical methods while being easier to deploy across different robot arms, making it practical for embodied AI systems.

PolicyIK uses reinforcement learning to solve inverse kinematics for robot arms with millimeter-level precision. Instead of relying on hand-tuned classical controllers, it trains a policy that converts end-effector pose commands into joint motions, handling the mathematical complexities of redundancy and singularities.

trainingapplicationsagents

GUILD: Guided Iterative Data Collection for Imitation Learning

Nov 3, 2026

Shivin Dass, Kevin Nguyen, Rohan Yelandur et al.

Active data collection guided by predictive attribution can reduce the amount of real-world robot teleoperation needed by 2-3x compared to unguided collection, because it focuses on collecting data that actually improves policy performance rather than just semantically relevant data.

GUILD is a method for collecting better training data for robot learning by actively deciding what to collect rather than passively gathering data. It uses vision-language models to suggest relevant tasks and environments, then evaluates which collected data actually helps the robot policy perform better, creating a feedback loop that guides future data collection toward more useful examples.

datatrainingapplications

MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection

Nov 3, 2026

Omar Nusrat

Even state-of-the-art language models make critical errors on specialized medical physics calculations—including omitting standard corrections—and show inconsistent error detection, suggesting LLMs need domain-specific verification before deployment in clinical settings where calculation mist...

MedPhysBench is a benchmark with 1,200 test items evaluating how well language models perform medical physics calculations and error detection across radiation therapy, imaging, and nuclear medicine.

evaluationsafetyapplications

PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data

Nov 3, 2026

Kewei Zhang, Juncheng Ma, Guodong Lv et al.

LLM agents can autonomously generate high-quality robot training data by building simulations and rendering photorealistic videos, achieving better generalization than human-collected data at a fraction of the cost.

A system where an LLM agent acts as a virtual teleoperator to generate robot training data. The agent receives a real robot scene, writes simulation code and controllers, then uses a video model to render photorealistic training episodes. The resulting synthetic data trains policies that match or exceed those trained on real teleoperated data, while costing 60-70% less than human teleoperation.

agentsapplicationsefficiency
agentsapplicationsreasoning

When to Assist and What to Suggest: Joint Gating and Memory-Grounded Hint Generation in One Compact Decoder

Oct 29, 2026

A. J. A. Jubayer Talukder, Mohammad Ariful Haque

A single small model can effectively decide when to assist and generate helpful hints by leveraging memory grounding, outperforming larger multi-model systems on speed while maintaining quality—memory context matters far more than retrieval or model size.

This paper tackles proactive assistance in conversations by combining gating (deciding when to help) and hint generation (what to suggest) into a single compact model. The authors show that a 1B parameter decoder can match larger two-model pipelines while being 1.31-1.76x faster, with memory-grounded hints being the key driver of quality.

efficiencyapplicationsreasoning

Gym-Anything for Robotics: Scaling physical simulations with digital agents

Oct 29, 2026

Pranjal Aggarwal, Sean Welleck

Using coding agents to generate simulations is a scalable way to create diverse, realistic robotics environments for training embodied AI, and combining multiple learning strategies (privileged information, self-learning, code-based policies) significantly improves performance on complex manipula...

This paper introduces Gym-Anything for Robotics, a framework where AI coding agents automatically create realistic robot simulations for training embodied AI models.

agentsapplicationstraining

Comparing Human Role-Players and LLM-Simulated Clients in Online Counselling Training: An Analysis of Counselling Patterns

Oct 29, 2026

Eric Rudolph, Philipp Steigerwald, Jens Albrecht

LLM-simulated clients can replicate many authentic counselling dynamics, but counsellors' communication patterns differ between human and AI interactions—suggesting LLMs are useful training tools but not perfect substitutes for human role-play.

Researchers compared how counselling students interact with human role-players versus LLM-simulated clients in training sessions. They analyzed conversation patterns at the sentence level to see if LLMs can authentically simulate clients and whether counsellors change their behavior based on who they're talking to.

applicationsevaluation

Advancing Clinical Genomic Analysis with a Skill-Centred Agent

Oct 29, 2026

Qiaoyu Zheng, Weike Zhao, Yanjie Fan et al.

Specialized AI agents built from curated domain skills can meaningfully augment expert decision-making in high-stakes clinical work, achieving better results than general LLMs by combining retrieval, orchestration, and executable tools.

ClinGeX is an AI agent that helps clinical genomicists analyze patient DNA sequences by orchestrating 105 specialized skills to interpret genetic variants and guide treatment decisions. In tests with real cases and clinical experts, it achieved 71.7% accuracy compared to 45.7% for competing systems, and significantly improved expert performance when used collaboratively.

agentsapplicationsreasoning

Ramanujan Challenge Results: AI-Assisted Evaluation Pipeline for Multi-Format Proofs

Oct 29, 2026

Alan Sorani, Ashvni Narayanan, Michael Shalyt et al.

Evaluating AI's mathematical reasoning requires specialized pipelines beyond single-pass LLM judgment—combining multiple verification methods dramatically improves accuracy on research-level problems.

Researchers evaluated AI models and formal tools on the Ramanujan Challenge, a benchmark of research-level number theory problems. They developed an AI-assisted pipeline to reliably judge complex proofs in multiple formats (natural language, code, formal proofs), reducing false positives from 23% to 6% compared to simple LLM evaluation.

evaluationreasoningapplications

TRACER, a time-to-event EHR foundation model for disease detection across health systems

Oct 29, 2026

Mara Pleasure, Ekaterina Redekop, Vedrana Ivezic et al.

Foundation models can screen for future disease, but evaluating them only on standard metrics (like AUROC) misses critical real-world problems: false-alert burden varies by disease and health system, so external validation at clinically appropriate thresholds is essential before deployment.

TRACER is a foundation model trained on 4.5M patient EHR records that predicts disease diagnoses 1-5 years in advance.

evaluationapplicationssafety

IndicIPR-QA: A Benchmark for Extractive Question Answering on Indian IPR Laws

Oct 29, 2026

M. Mohaiminul Islam, Muhammad Abulaish

Building legal QA systems for low-resource languages requires domain-specific benchmarks and models—simple retrieval methods fail dramatically (20% F1) compared to fine-tuned legal BERT (85% F1) on Indian IPR law questions.

This paper introduces IndicIPR-QA, a benchmark dataset of 4,462 question-answer pairs for extractive QA on Indian intellectual property law across seven statutory acts. The dataset was validated through automatic checks, LLM evaluation, and human review, with experiments showing that fine-tuned legal BERT models significantly outperform simple keyword matching on this task.

evaluationdataapplications

AI in the EGI e-infrastructure -- Past experiences and future strategies

Oct 29, 2026

Gergely Sipos, Sergio Andreozzi, Marco Rorro et al.

The AI landscape for research has shifted from building general models to adapting existing AI tools for specific scientific domains—EGI's strategy emphasizes infrastructure, skills, and trust as the foundation for this practical, domain-focused AI deployment.

EGI Federation shares 15+ years of experience deploying AI across European research infrastructure, from early digital twin projects to modern domain-specific applications. The paper outlines EGI's 2026-2030 strategy focusing on three pillars: AI-ready infrastructure connecting compute and data, workforce skills development, and trustworthy AI operations for high-risk research environments.

applicationstraining

Agentic University: Collaborative Agents for Personalized Learning

Oct 29, 2026

Jingru Fan, Tianle Zhou, Yunjie Zhou et al.

Multi-agent systems can coordinate around learner needs more effectively when learners actively participate in organizing their own learning—not just consuming content, but helping shape what agents do next.

This paper presents Agentic University, a system where multiple AI agents collaborate with learners to create personalized learning programs. The system connects learning goals, activities, and assessments while using learner feedback to adapt what comes next. Students using this approach showed significantly better learning gains and reported feeling better supported.

agentsapplicationsalignment

FIRME: Registry-Anchored Memory Graphs for Evolving, Heterogeneous Investor Knowledge

Oct 29, 2026

Yunbai Zhang, Yunyi Gao, Han-Tai Shiao

For AI systems that need to remember and reuse knowledge over time, maintaining a canonical registry with persistent aliases and source tracking prevents fragmented relationship histories and lets evidence accumulate reliably.

FIRME is a memory system for financial AI that tracks investor knowledge across time and sources by maintaining a canonical registry of relationships. It reconciles different ways of describing the same observation (e.g., "Apple stock" vs "AAPL") and links investor discussions to cash-flow data, while preserving where each piece of information came from and when.

dataapplications

KPPax: preprocess differentiable kinetic ODE for machine learning in atmospheric chemistry

Oct 27, 2026

Wenqing Peng, Michael Boy, Zhi-Song Liu

By making atmospheric chemistry models differentiable, KPPax enables automatic computation of parameter sensitivities and hybrid physics-ML models without hand-coded adjoints—reducing emission estimation errors by 2.4–5× in demonstrated applications.

KPPax converts atmospheric chemistry models from KPP format into differentiable JAX code, enabling automatic differentiation and machine learning integration.

trainingefficiencyapplications

MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices

Oct 27, 2026

Kailai Feng, Haoxiang Rao, Yuxiang Wei et al.

Mobile image models need different evaluation criteria than server models—this benchmark shows that expanding task coverage matters more than further optimizing deployment efficiency, and specific capability gaps (like text replacement) should be prioritized over broad improvements.

MoI-Bench is a benchmark for evaluating image generation and editing models on mobile devices, with 1150 samples across 64 subtasks defined by combinations of 16 operations (like removal, replacement) and 8 visual targets (portraits, text, etc.).

evaluationefficiencyapplications

GAS-PFN: Graph-Aware Spatiotemporal Prior-Data Fitted Networks for Zero-Shot Traffic Forecasting

Oct 27, 2026

Joseph Natter, Long Vu, Yifan Zhang et al.

By explicitly encoding road network structure into the training process and using a two-stage inference design, you can build traffic forecasters that work on new networks without retraining—a major practical advantage for deploying to evolving city infrastructure.

This paper introduces GAS-PFN, a neural network that forecasts traffic flow without needing to be retrained for new road networks. It learns from synthetic traffic patterns during training, using graph structure (road topology) to understand how traffic moves between connected sensors.

reasoningarchitectureapplications

Zero-Shot Multimodal Event Spotting in Soccer Broadcasts

Oct 27, 2026

Sadia Kamal, Mahmudul Hasan, Tony Braskich

General-purpose multimodal models can effectively spot sports events without fine-tuning by leveraging compact visual representations and aligned text, suggesting that domain-specific training may be unnecessary for event detection tasks.

This paper presents a zero-shot approach for detecting and localizing soccer events in video broadcasts using multimodal models, without requiring task-specific training. By combining visual storyboards and commentary transcripts processed by a general-purpose model, it achieves strong performance while reducing computational cost by 72% compared to frame-by-frame analysis.

multimodalapplications

LCM-Promo: An LLM-Based Uplift Model for Promotion Allocation

Oct 27, 2026

Nesta Midavaine, Floris Fok, Paulo Mendes et al.

Fine-tuning LLMs on commerce transaction data can outperform traditional segmentation approaches for promotion targeting by learning individual customer price sensitivity directly from production logs.

LCM-Promo is an LLM fine-tuned on transaction data to decide which customers should receive promotions and at what discount level. Instead of treating promotion decisions as simple yes/no choices, it learns how different customers respond to different discount levels, then uses this to allocate promotions more efficiently.

applicationstrainingevaluation

Pneuma: A Large-Scale Real-World Time Series Benchmark for Commercial Aircraft Predictive Maintenance

Oct 27, 2026

Jiafeng Lin, Yutian Zhang, Huan Zhang et al.

Real-world predictive maintenance is harder than lab benchmarks suggest—even strong models fail on subtle faults in actual aircraft data, and the dataset's class imbalance and cross-aircraft variance are critical challenges developers must address.

Pneuma is the first public large-scale benchmark dataset for aircraft predictive maintenance, containing over 920 million time series measurements from real commercial aircraft.

evaluationdataapplications

Patient-specific rubric-based evaluation of large language models across longitudinal cancer care

Oct 26, 2026

Wei-Wei Zhang, Pengkai Wang, Yan Li et al.

LLMs need phase-specific evaluation and safeguards for medical AI—strong performance on one clinical task doesn't guarantee reliability across the full patient journey, and the types of errors models make shift dramatically depending on the care stage.

This paper introduces NPC-RubricBench, a benchmark for evaluating how well large language models handle cancer care across multiple treatment phases rather than single encounters.

evaluationsafetyapplications

Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy

Oct 26, 2026

Yifei Yao, Yude Zou, Guanbo Jiao et al.

Frontier language models can control robots competitively with specialized policies on many tasks, but hybrid approaches that let the agent delegate to specialized policies achieve better overall performance—suggesting future robot agents should be designed to strategically invoke specialized c...

This paper evaluates GPT-6-Astra, a frontier multimodal language model, as a robot control agent using EBench. The study compares direct control (agent generates commands directly) versus a harness approach (agent can invoke specialized policies).

agentsmultimodalapplications

Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent

Oct 26, 2026

Benjamin Pikus, Daniel Saunders, Mircea Mironenco et al.

Pretrained language models can be efficiently adapted for robot control through simulation-based training alone, enabling strong real-world performance and compositional generalization without collecting real robot data.

ZAP is a robot control system that combines a pretrained vision-language model with simulation-based training to enable robots to understand tasks and execute them without real-world training data. The approach achieves 96.7% success on real-world manipulation tasks while being 3x faster and 17x cheaper than GPT-4 Astra, and generalizes to unseen objects, goals, and robot embodiments.

trainingapplicationsreasoning
evaluationapplicationsdata

Monetizing Intelligence: Advertising and Subscription Design on LLM Platforms

Oct 23, 2026

Yuhang Ji, Fasheng Xu, Tianjun Feng

Platform design choices about ads and pricing affect everyone—not just ad viewers. A middle-tier ad-supported plan can fund better service for some users while actually making premium plans more expensive by degrading alternatives.

This paper analyzes how LLM platforms should balance advertising, subscription pricing, and service quality. It shows that ad-supported tiers can expand access but create utility gaps between users, that moderate ad caps can improve consumer welfare, and that cheaper compute paradoxically increases total ad revenue by making advertised content more valuable.

applicationsalignment

If at First You Don't Succeed, Backtrack: Lightweight Recovery for Frozen VLAs

Oct 23, 2026

Thomas Landeg, Thierry Tambe

You can add robust failure recovery to frozen robotic VLAs with minimal overhead by training a small failure detector on the model's existing representations and using simple motion replay—no retraining or new data needed.

Backtrack is a lightweight failure recovery system for frozen Vision-Language-Action (VLA) robotic models that detects when a task will fail and automatically retraces the robot's recent motion to retry from a safer state.

efficiencyagentsapplications

Cost-Aware Abstention for LLM-Based IT Ticket Classification

Oct 22, 2026

Sachin Gupta

For high-stakes classification tasks like IT ticket routing, knowing when to abstain and defer to humans is more valuable than always routing automatically—and simpler models with proper calibration often outperform LLMs when costs of errors are asymmetric.

This paper evaluates when IT support tickets should be automatically routed versus sent to human triage, using cost-aware abstention.

evaluationapplicationssafety

How Ground Truth Shapes Retrieval Scores: Evidence-Aware Evaluation of a Retrieval-Augmented Generation System for Breast Lesions

Oct 22, 2026

Miguel Mochizuki Silva, Samuel Ismael, Mateus C. Dias et al.

Retrieval evaluation scores are highly sensitive to annotation completeness—expanding ground-truth evidence annotations improved reported metrics without changing what the system actually retrieved, suggesting evaluation protocols must account for annotation coverage to avoid misleading perform...

This paper evaluates a retrieval system for a clinical AI tool that finds relevant medical evidence about breast lesions. The researchers tested different ways to chunk documents and rank results, finding that a hybrid approach combining multiple retrieval methods achieved strong performance.

evaluationapplicationsdata

VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act

Oct 22, 2026

Robin Quériaux, Solveig Bachellery, Léa Déléris et al.

When deploying LLMs in regulated settings, a user-friendly dashboard that translates technical benchmarks into compliance evidence can cut review time in half and make responsible AI assessment accessible to non-engineers.

VERA is an open-source dashboard that helps organizations assess whether large language models meet EU AI Act requirements before deployment. It automates twelve measurable responsible AI checks, tracks energy consumption, and presents results to non-technical reviewers—reducing assessment time from 56 to 30 seconds and scoring 4.36/5 on usefulness in a bank deployment.

safetyevaluationapplications

Action Learning for Scientific Simulations

Oct 22, 2026

Sarah Liaw, Yuanqi Du, Yilun Du

Action learning provides a unified, reusable objective for reasoning about dynamical systems from trajectory data—enabling path ranking, control, and inference without retraining for each new task.

This paper introduces LEAP, a method for learning stochastic actions from trajectory observations in dynamical systems. By anchoring unknown actions to a reference process and learning corrections through trajectory comparison, LEAP creates a differentiable objective that captures both local dynamics and long-term dependencies.

reasoningtrainingapplications

Social-HUMINT: An LLM-Augmented Unified Framework for Measuring Social Media Exposure

Oct 22, 2026

Pellegrino Casoria, Simon Pietro Romano

LLMs can infer sensitive personal attributes from public social media with 97% accuracy, enabling precise targeting for social engineering—defenders need systematic frameworks to measure this exposure before attackers exploit it.

This paper presents a framework for measuring how vulnerable people are to social engineering attacks based on what can be inferred about them from public social media. Using LLMs, researchers extract personal information from posts, infer hidden attributes, and predict which manipulative messages would work on each person.

safetyevaluationapplications

Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research

Oct 21, 2026

Houde Qian, Zheng Jiang, Wenyi Li et al.

LLMs can generate plausible trading code and features, but current models fail to produce strategies that generalize beyond their training period—a critical gap between research-phase performance and real-world deployment.

Quant-Arena is a benchmark that tests whether LLMs can complete the full quantitative trading research loop—from hypothesis to strategy—on real market data. Models develop trading strategies using 2022-2023 data, then their frozen strategies are evaluated on unseen 2024 data with realistic trading costs and market impact.

evaluationagentsapplications

AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation

Oct 21, 2026

Likui Zhang, Zhe Zhu, Zhihao Zhan et al.

Breaking long-horizon robotic tasks into atomic subtasks and using semantic-aware gating for credit assignment enables more efficient RL training of vision-language-action models, with practical real-world transfer via simulation.

AtomicRL improves vision-language-action (VLA) models for robot manipulation by using reinforcement learning with better credit assignment. It breaks long tasks into atomic subtasks, uses semantic-aware gating to prevent credit contamination between subtasks, and adaptively focuses learning on bottleneck skills.

agentsapplications

Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs

Oct 20, 2026

Noam Rotstein, Yoad Tewel, Chen Tessler et al.

You can now generate video and 3D animation together for any character rig—not just humans—and automatically place that animation in a 3D scene with correct camera positioning, enabling practical animation workflows from text or video.

This paper presents a model that generates both realistic video and replayable 3D motion simultaneously for any character rig. By keeping 3D motion in a separate model connected to a video generator through cross-attention layers, it avoids adding billions of parameters while supporting arbitrary skeletons.

multimodalapplications

GBClaw: An agentic AI framework for atomistic grain boundary research and mechanistic discovery

Oct 20, 2026

Yazhuo Liu, Kunqing Ding, Xiaoguo Gong et al.

Agentic AI can tackle domain-specific materials science by combining LLMs with specialized simulation tools and memory systems, enabling discovery of mechanistic insights that require both high-throughput screening and deep physical reasoning.

GBClaw is an AI framework that combines large language models with specialized tools to automatically investigate grain boundaries in materials. The system orchestrates complex simulations and analysis to understand how grain boundaries deform, discovering that defect structures significantly affect material strength—work that would be difficult for humans to do manually.

agentsapplicationsreasoning

Semantic Observability for Autonomous Systems Diagnostics

Oct 20, 2026

Jitendra Solanki, Anand Todkar, Ziran Min

By representing logs as a connected graph of infrastructure and events rather than flat text, autonomous agents can efficiently isolate failure causes and explain them in code-level detail—cutting through noise that overwhelms traditional log analysis.

This paper presents Semantic Observability, a multi-agent system that automatically diagnoses failures in distributed systems by converting unstructured logs into a queryable graph database, then using causal reasoning and LLMs to pinpoint root causes and suggest fixes. It reduces manual work for engineers investigating system outages.

agentsreasoningapplications
applications
reasoning

Discovering state-of-the-art quantum circuits with large language models

Oct 15, 2026

Andi Gu, Alek H. Kemeny, Susanne F. Yelin et al.

LLMs can effectively co-design quantum algorithms by proposing circuit structures that an evaluator scores, discovering practical improvements over hand-optimized quantum code without requiring the LLM to understand quantum mechanics deeply.

Researchers used large language models working with an independent evaluator to discover optimized quantum circuits for various computational tasks.

reasoningarchitectureapplications

FDE-Gym: The Last Mile of AI Delivery

Oct 15, 2026

Dongxu Yang, Hongda Yan, Renxin Wang et al.

Current AI models achieve only 0.

FDE-Gym is a benchmark of 95 real-world delivery tasks where AI agents must clarify ambiguous customer requirements, work in live systems, and discover unstated constraints—mirroring the work of forward-deployed engineers.

evaluationagentsapplications

Lean Graph Search for Formal Theorem Proving

Oct 15, 2026

Yihan Xu, Jia-Chen Gu, Junyi Zhang et al.

Graph-based lemma retrieval that follows dependency chains improves theorem proving success rates by up to 40%, showing that understanding how mathematical concepts depend on each other is more powerful than keyword matching alone.

This paper presents LeanGraphSearch, a retrieval system for formal theorem proving that uses dependency graphs to find relevant lemmas. Instead of just matching keywords, it traces multi-hop connections between mathematical declarations in Lean's Mathlib library, uncovering supporting lemmas that help prove theorems more effectively.

reasoningevaluationapplications

Training Reasoning Models to Generate Diverse Scientific Ideas

Oct 15, 2026

Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi et al.

Set-level RL with distance-weighted scoring teaches models to generate diverse ideas without sacrificing quality—a 8B model beats much larger models on ideation tasks by optimizing for both novelty and feasibility together.

This paper trains LLMs to generate diverse, high-quality scientific ideas from research papers using set-level reinforcement learning. Instead of optimizing individual ideas, the model generates K structured idea cards where each new idea is weighted by its distance from previous ones, encouraging diversity.

trainingreasoningapplications

AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

Oct 15, 2026

Yuruo Wu, Kaihang Pan, Kai Shen et al.

For audio-video generation, evaluating consistency at the event level—rather than treating entire clips as single units—better captures real-world complexity and aligns more closely with human judgment.

This paper introduces AVC-Reward, an event-centric evaluation framework for assessing audio-video consistency in generated content. Unlike existing metrics that treat clips holistically, it breaks down complex scenes into individual audio and visual events, checks their temporal alignment and attribute consistency across modalities, and aggregates results into a three-level metric.

evaluationmultimodalapplications

A team of Gemini 4 Argon agents beats Minecraft

Oct 15, 2026

Maxence Faldor, Satinder Singh, Roberta Raileanu

Large language models can now autonomously plan, code, debug, and coordinate across thousands of dependent steps in complex environments—showing progress toward AI agents that can handle enterprise-scale software engineering tasks.

Four AI agents (Gemini 4 Argon) autonomously completed Minecraft's final boss challenge by coordinating through chat and shared memory, writing 2,783 programs over 27 hours without human intervention. This demonstrates AI's ability to maintain coherent long-horizon plans and collaborate on complex multi-step tasks similar to real-world software engineering.

agentsreasoningapplications

MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support

Oct 15, 2026

John Mendonça, Daniel Tschernutter, José Pombal et al.

Evaluating mental health AI requires both interpretable rubrics from domain experts and realistic patient simulations; current models' biggest weakness is misjudging how much guidance to give in different situations.

MindEval2 is an evaluation framework for testing how well AI chatbots provide mental health support. It includes clinician-designed rubrics that break down response quality into clear, actionable criteria, and a realistic patient simulator with memory that tracks conversations over time.

evaluationsafetyapplications

AI Consilium: a Benchmark of LLM Clinical Multidisciplinary Second Opinions

Oct 15, 2026

Alexey A. Bokov, Ivan I. Panin, Nikolay B. Nechaev et al.

LLMs can provide clinically useful second opinions when structured to reason step-by-step and collaborate across specialties, but model choice matters more than output format alone—and physician evaluation is essential for validating AI medical advice.

Researchers created a benchmark to evaluate how well large language models can provide clinical second opinions by having multiple AI models role-play different medical specialists and collaborate on complex diagnostic cases.

evaluationapplicationsreasoning