Recent AI research papers with accessible summaries. Updated daily from arXiv, summarized for developers who don't read papers regularly.
Daniel Reichenpfader, Kwaku Appiagyei, Manovah Nadarajah et al.
LLM agents can efficiently draft data schemas from unstructured medical text and extract structured information at scale, but human expert validation remains essential for ensuring accuracy and clinical relevance.
Researchers used AI agents to automatically extract and organize medical data elements from thousands of cancer research abstracts. Starting with nearly 73,000 candidate data fields, they refined them to 53 validated fields and successfully extracted information from 79% of abstracts, with human reviewers confirming 83% were relevant.
Ethan Lin, Akanksha Sarkar, Ziang Liu et al.
AI agents can automatically synthesize effective semi-supervised learning pipelines, but the bottleneck isn't code generation—it's reliably selecting which generated programs actually work best.
This paper studies whether AI agents can automatically design semi-supervised learning (SSL) pipelines when labeled data is scarce. The authors show agents can discover code combining SSL techniques that outperform human baselines by up to 21.8%, but reveal a critical weakness: selecting which generated programs to use drops performance by 47%.
Federico Pennino, Andrea Gurioli, Maurizio Gabbrielli et al.
For code model pre-training, you can now query structural properties (AST patterns, definitions, etc.) across massive corpora in seconds without downloading everything—enabling faster, more targeted dataset curation.
SWHForest is a framework that creates a searchable structural index of code files without storing source code, letting researchers efficiently filter and curate large code corpora before downloading. Instead of downloading millions of files to find what they need, users can query the index to identify matching files first, reducing downloads by 46-72%.
Linrong Cai, Taiming Lu, Haozhe Chen et al.
Migrating ML code between frameworks and hardware is hard enough that it serves as a meaningful benchmark for AI agents, and today's best models still fail on a quarter of real-world migration tasks.
XStack is a benchmark that tests AI agents' ability to migrate machine learning repositories from PyTorch (NVIDIA GPUs) to JAX (Google TPUs). Researchers created a grading system based on published results to evaluate whether migrated code works correctly.
Chengyang He, Wenlong Zhang, Koduvayur Subbalakshmi et al.
Language models can sound confident about temporal patterns in data without doing proper statistical testing—this benchmark reveals they fail mainly at selecting relevant evidence, not at running tests, and a structured retrieval-then-test approach significantly improves accuracy.
This paper introduces STRATA, a benchmark for evaluating language models on temporal clinical hypothesis testing. Rather than just grading final answers, it evaluates every step: choosing which patient records matter, selecting the right statistical test, and interpreting results.
M. Mohaiminul Islam, Muhammad Abulaish
Building legal QA systems for low-resource languages requires domain-specific benchmarks and models—simple retrieval methods fail dramatically (20% F1) compared to fine-tuned legal BERT (85% F1) on Indian IPR law questions.
This paper introduces IndicIPR-QA, a benchmark dataset of 4,462 question-answer pairs for extractive QA on Indian intellectual property law across seven statutory acts. The dataset was validated through automatic checks, LLM evaluation, and human review, with experiments showing that fine-tuned legal BERT models significantly outperform simple keyword matching on this task.
Federico Lucca
Before deploying AI systems, measure ground truth carefully and optimize for the actual business metric, not just accuracy—a perfectly accurate classifier can fail catastrophically if it ignores rare but important cases.
This paper evaluates email routing systems for a personal multilingual inbox using rigorous measurement before training.
Sy-Tuyen Ho, Anirudh Satheesh, Soham Chawathe et al.
Low-frequency visual signals embedded in training images can reliably detect if a T2I model was fine-tuned on that dataset, enabling practical dataset-use auditing without degrading image quality or requiring complex per-image optimization.
SpectralCanary is a method for auditing whether text-to-image models were trained on specific datasets by embedding a hidden signal into images before release. It uses a smooth low-frequency mask to blend a canary image into training data, then checks if that signal appears in suspect model outputs. The approach achieves 91-92% detection rates while keeping images visually unchanged.
Tianjiao Yu, Zhilin Ou, Yifan Shen et al.
Co-evolving data generation and policy training through shared failure diagnosis and staged learning signals can dramatically improve robot learning efficiency and robustness to real-world variations.
CoDA is a framework where a robot learning system and a data generation engine improve together by sharing a memory of failures and solutions. A vision-language critic diagnoses why policies fail, and this feedback guides both the creation of challenging training environments and the design of better learning signals.
Dixi Yao, Tahseen Rabbani, Tian Li
Random projection is a simple, practical alternative to differential privacy for protecting RAG datastores: it maintains retrieval quality while defending against data extraction attacks at lower computational cost.
This paper proposes using random projection to protect sensitive data in RAG systems. By applying the same random transformation to both stored knowledge embeddings and user queries, the method preserves search accuracy while preventing attackers from extracting private information from the datastore—without the computational overhead of formal privacy guarantees.
Joseph Lee, Yanjing Li, Bo Qiang et al.
Natural language is now a practical interface for protein design—you can describe desired protein properties in text and get back 3D structures, with experimental validation showing it works for real-world targets.
Prompteina is an AI model that generates protein structures from natural language descriptions, similar to how text-to-image models work. Trained on 1.8M protein structures with detailed captions, it can design proteins with specific structural and functional properties without needing separate input formats for each goal.
Oktie Hassanzadeh, Dharmashankar Subramanian
LLM-based semantic query engines are non-deterministic even at temperature 0—benchmark comparisons based on single runs can be misleading, and you need multiple executions or voting strategies to get reliable results.
This paper reveals that semantic query engines using LLMs produce inconsistent results across multiple runs, even at temperature 0. By re-executing 1.2 million model calls across three benchmarks, the authors show that 0.07-10.2% of individual rows change their output between runs, and up to 66% of final answers differ.
Lyuxin David Zhang, Eric Wong, Surbhi Goel et al.
You can select effective training data for LLMs using only output-layer gradients from forward passes, cutting computation costs by ~10× compared to full-gradient methods without sacrificing downstream performance.
This paper proposes LESSER, a method for selecting high-quality training data for large language models by using only output-layer gradients instead of full-parameter gradients. By leveraging cheaper forward passes rather than expensive backward passes, LESSER reduces computational cost by 9.7× for supervised fine-tuning while maintaining performance comparable to full-gradient selection methods.
Yiming Huang, Yujie Zeng, Vijay Prakash Dwivedi et al.
HGR enables sequence models to generate chemically valid molecules with perfect validity while capturing complex molecular topology, achieving top performance on generation and property prediction tasks without the computational cost of explicit higher-order encodings.
This paper introduces Higher-order Grammar Representation (HGR), a new way to represent molecules that captures complex structural features like ring systems by converting them into sequences of grammar rules. Unlike existing methods that struggle with computational overhead, HGR makes these structures compatible with standard sequence models while guaranteeing valid molecules.
Jordan L. Cahoon, Chloe O. Stanwyck, Sulaiman Somani et al.
Automated benchmark generation validated by domain experts enables continuous evaluation of clinical AI systems, exposing real gaps (like multi-document synthesis) that static benchmarks miss.
Researchers created BRIE, an automatically-generated benchmark for testing how well AI systems retrieve and synthesize information from patient medical records.
Sriram Kannan, Swetha Saseendran, Vishnu Vardhan Reddy Kandi et al.
Structured event graphs from legal documents improve reasoning tasks like classification and QA, but only when you already have the right documents—they don't help with initial retrieval.
ARGUS builds structured Event Knowledge Graphs from employment-discrimination legal complaints using a 5W1H schema and LLMs. The system extracts facts, organizes events with temporal and causal relationships, and merges them into document-level graphs. Testing shows these graphs improve claim classification and help answer legal questions when relevant documents are already retrieved.
Rawan El Ghali, Umm Kulsoom, Anas Madkoor et al.
Multiple-choice scoring masks AI model failures in language understanding—QuranicMMLU shows models score 24 percentage points higher on multiple-choice than open-ended tasks, meaning benchmark design significantly impacts what we learn about model capabilities.
QuranicMMLU is a benchmark for testing how well AI models understand Quranic Arabic across five linguistic areas: sound, word structure, grammar, meaning, and context.
Sho Kawano, Zehang Richard Li, Paul A. Parker
When evaluating AI systems across domains with limited labels, borrowing information across domains through prediction-powered smoothing gives more accurate performance estimates than evaluating each domain independently.
This paper addresses how to accurately evaluate AI systems across different domains (like task types or conversation types) when you can only label a small sample. It proposes prediction-powered smoothing, which combines AI predictions with limited labels to get better estimates for each domain, and a validation method to choose between different estimation approaches.
Masahiro Kato, Daiki Honma, Taka Kato
Businesses can now quantify the ROI of appearing in generative AI outputs by combining visibility data with causal inference, enabling marketing decisions in an AI-driven world where traditional metrics don't apply.
This paper introduces a framework to measure how generative AI systems (like ChatGPT) affect business outcomes when companies optimize their visibility in AI-generated answers.
Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang et al.
No single causal discovery method generalizes across different benchmark types; evaluation results depend heavily on which test scenario you use, making diverse benchmarks essential for fair comparison.
CausalArena is a comprehensive benchmark for evaluating causal discovery methods—tools that learn cause-and-effect relationships from data. It addresses a critical problem: existing benchmarks use different test scenarios, making it hard to compare methods fairly.
Yuxiao Li, Keke Hu, Santiago Mazuelas et al.
Synthetic wireless signals generated by GANs can effectively replace expensive real-world data collection for training wireless sensing models, reducing labeling costs while maintaining physical realism.
This paper introduces IIns-GAN, a deep learning method that generates realistic labeled wireless signals for training machine learning models.
Dain Kim, Eungi Cho, Kyumin Kim et al.
Open-source models can match much larger models at multi-step API calling by training on execution-verified data—showing that smart data synthesis matters more than model size for tool-use tasks.
This paper addresses a real-world problem: open-source AI models struggle when they need to call multiple government APIs in sequence to complete tasks. The authors create KOPA-Bench, a benchmark of 145 real Korean government API tasks, and introduce EDGE, a method that learns which API outputs can feed into other APIs by actually testing them live.
Dewu Zheng, Ruizhe Ye, Yanlin Wang et al.
Filtering training data for code-fixing tasks at both trajectory and step levels beats training on all successful examples—quality and relevance of supervision matter more than quantity.
SWE-Prime improves how AI models learn to fix software bugs by being smarter about which training examples to use. Instead of training on all successful bug fixes, it filters trajectories (problem-solving paths) at two levels—first selecting high-quality complete solutions, then identifying which individual steps within those solutions are actually worth learning from.
Maayan Sharon, Tom Hope
Retrieval systems need to understand different types of scientific inspiration—not just finding similar papers, but finding papers that generalize ideas, concretize them, or solve stated problems.
RATIO is a benchmark for retrieving scientific papers that can inspire research at different levels of abstraction. It defines three types of retrieval: finding approaches to solve problems, generalizing to broader concepts, or narrowing down to concrete implementations.
Yucheng Jiang, Zora Zhiruo Wang, Ruishi Chen et al.
By converting raw computer activity into structured task models with goal hierarchies and control flow, TMI enables AI agents to learn realistic work procedures and organizations to audit and reuse task knowledge from employee activity traces.
This paper presents Task Model Induction (TMI), a method that automatically discovers and structures how people actually work on computers by analyzing screenshots and input logs.
Fengqing Jiang, Yite Wang, Boyi Liu et al.
Tool-use capabilities in language models improve significantly when trained during mid-training with targeted synthetic data, rather than waiting until post-training—similar to how math and reasoning skills benefit from dedicated training phases.
MidTool is a data synthesis pipeline that creates training data for teaching language models to use tools effectively during mid-training (the stage between pretraining and fine-tuning).
Zhelun Wu
When combining evidence from multiple sources, separate the task of interpreting each source from aggregating those interpretations—use structured tuples and calibrated scoring rather than simple concatenation and vote counting.
This paper separates evidence interpretation from decision aggregation in multi-source reasoning systems. Instead of concatenating sources into one prompt, the authors propose a structured evidence tuple (hypothesis, reliability, rationale, provenance) and show how to properly combine interpretations using calibrated log-likelihood ratios.
Hao Yan, Lisa Pilgram, Dan Liu et al.
You can now generate realistic synthetic tabular health data from multiple heterogeneous sources by standardizing them statistically first, then using diffusion transformers to learn and reproduce their patterns.
This paper presents a method for generating synthetic health data from multiple different database tables with varying structures. It works in two stages: first converting diverse tables into a standardized statistical format, then using a diffusion transformer to learn patterns and generate new synthetic tables.
Donna Hooshmand, Shubham Shahi, Cameron Barrie et al.
Automating semantic schema construction lets non-technical users query databases without expert help, and TYTAN's hybrid symbolic-LLM approach achieves production-ready accuracy by knowing when to ask humans for clarification.
TYTAN automatically builds semantic schemas for relational databases by combining symbolic analysis with LLM inference to identify entities, relationships, and data roles. It asks targeted questions when ambiguous, achieving 100% coverage and correctness on real-world databases—eliminating the manual work that currently bottlenecks data analysis tools.
Yiting Zheng, Cheng Fang, Anthony Donofrio et al.
Using a unified graph representation of reactions instead of separate reactant/product graphs enables better learning of chemical transformations, leading to more accurate yield predictions and a foundation model that could generalize across diverse reaction types.
RxnCLF is a self-supervised learning framework that represents chemical reactions as unified graphs (condensed reaction graphs) to better capture how molecules transform.
Bing Yan, Gregory Wolfe, Stefano Martiniani et al.
Instead of searching papers, search claims: AskChem lets you find specific findings with built-in source verification, reducing manual work for literature synthesis and improving AI agent reliability by 12% on citation accuracy.
AskChem is a search infrastructure that converts chemistry papers into atomic, provenance-tracked claims rather than returning full documents. Scientists and AI agents can search across 2.4M claims from 147K papers, with each claim linked to its source DOI and exact evidence location, enabling faster synthesis of findings across multiple papers.
Haomin Qi, Xingliang Wang, Xuanqi Gao et al.
By mining repository history and reconstructing code states, you can generate verified, executable coding tasks at scale—reducing the manual effort of creating training data for code-generation agents while maintaining realistic development scenarios.
Change2Task automatically converts pull requests from repository history into executable coding tasks for training AI agents.
Anjian Li, Ryne Beeson
Collecting intermediate solver iterations during optimization—not just final solutions—significantly increases training data for learning-based optimization without additional computational cost, with theoretical guarantees on generalization.
This paper addresses data scarcity in training models that generate initial guesses for optimization problems. The authors propose collecting intermediate solver steps (k-neighborhoods) alongside final solutions to increase training data without extra computation.
Nethmi Muthugala, Supryadi, Surangika Ranathunga et al.
LLMs trained primarily on Western data misalign with non-Western cultural values; this work provides a replicable framework for embedding country-specific values in low-resource languages through survey-grounded datasets and targeted fine-tuning.
This paper introduces LKValues, a resource suite for aligning large language models with Sri Lankan cultural values. The authors surveyed 205 Sri Lankans to identify 40 key societal values, created a 150k-instance instruction dataset in Sinhala and English, and built an evaluation benchmark.