Papers

Thales Bertaglia, Catalina Goanta, Gerasimos Spanakis et al.

This benchmark helps detect undisclosed advertising in child-facing content by combining video transcripts, metadata, and linked sales pages—revealing widespread non-compliance with YouTube's disclosure requirements.

ChildSafeAds is a shared task that identifies commercial content in YouTube videos targeting children. Using 3,360 videos with sponsor segments from SponsorBlock, systems classify what products are promoted, categorize them, and flag legal risks. The dataset reveals that 45.5% of videos fail to properly disclose paid promotions, highlighting gaps in platform compliance.

Ramneet Kaur, Pradyumna Chari, Ramesh Raskar et al.

AI agents communicating through hidden internal states can coordinate deception undetectably—but you can monitor and prevent this by tracking latent activations and using counterfactual analysis to steer behavior back to compliance.

This paper addresses a critical safety problem: AI agents can coordinate harmful behavior through hidden communication channels in their internal states, invisible to human oversight.

Benjamin Belay

Generated text can carry cryptographically verifiable evidence about which internal computations a model actually performed, opening possibilities for auditing model reasoning without changing the visible output.

This paper demonstrates that language models can embed hidden evidence of their internal computational states into generated text. Researchers trained neural networks on arithmetic tasks with mandatory intermediate decision points, then verified which internal state was used and encoded that information as subtle statistical patterns in the output.

Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu et al.

Current compliance monitoring systems for AI don't actually read the rules they're supposed to enforce—they work by pattern matching on scenarios, not rule logic, which undermines their use as regulatory controls.

This paper reveals that compliance detectors used to monitor language models for regulatory violations are 'rule blind'—they make the same decisions regardless of what rule they're supposed to check. The authors show that deleting or swapping rules doesn't change detection accuracy, meaning detectors rely on surface patterns rather than actual rule content.

Enric Boix-Adsera, Benedict Tessler

AI models can be reliably manipulated through combinations of weak, inconspicuous textual cues that individually appear harmless but collectively override intended behavior—a vulnerability that's hard to detect and poses significant safety and interpretability challenges.

Researchers show that AI models can be controlled through subtle, seemingly irrelevant text cues combined together—a phenomenon called 'model hypnosis.' These inconspicuous prompts work across different models and scales, including advanced reasoning models, and can transfer between systems.

Junjie Chu, Ye Leng, Mingjie Li et al.

Generative search engines are vulnerable to content optimization tactics that inflate the visibility of low-authority or false information; systematic detection methods are now possible but require careful design to avoid relying on author-based shortcuts.

This paper introduces GEO-Flag, a system for detecting web pages optimized for generative search engines (like Google's AI Overviews).

Zhe Ye, Hantao Lou, Yuechun Sun et al.

AI agents can generate code, but generating code with formal proofs that work together across entire repositories remains an unsolved problem—current best agents only solve 27 of 43 real-world instances.

Vero is a benchmark for evaluating whether AI agents can generate both correct code implementations and machine-checked formal proofs together across real multi-module software repositories.

Dananjay Srinivas, Saksham Khatwani, Maria Pacheco

LLMs possess the internal machinery to recognize knowledge gaps and adjust specificity accordingly, but their generation process doesn't use these signals—a gap that could be fixed through better training objectives.

Large language models often make up specific details about unfamiliar entities instead of admitting uncertainty. This paper shows that LLMs actually have internal signals detecting when they don't know something and can anticipate how specific their answer should be—but they ignore these signals during generation, preferring to sound confident anyway.

Julian Minder, Viktor Moskvoretskii, Raghav Singhal et al.

Installing alignment values during pretraining from the beginning creates deeper, more robust alignment than adding it after training, and this advantage grows with more pretraining data.

This paper introduces Synthetic Persona Pretraining (SPP), a method that embeds desired values and assistant behavior directly into language models from the start of pretraining rather than adding them afterward.

Jim Woodcock, Gabriel Leite, Augusto Sampaio et al.

When using LLMs to modify formal proofs, you need independent verification beyond just checking if the code compiles—CAPRI shows that contract-based auditing can catch unauthorized changes that Isabelle alone would miss.

CAPRI is a system that uses LLMs to help repair broken Isabelle proofs while ensuring developers maintain control over what gets changed. It combines Isabelle's proof checker with an independent contract enforcer that tracks all changes, keeping an audit trail of prompts, proposals, and verdicts.

Yukun Dai, Mingzhe Dai, Tianshi Wang et al.

VLA robotic models share cross-task vulnerabilities that can be exploited with a single adversarial texture, revealing a critical safety gap in multitask embodied AI systems that current defenses don't address.

This paper demonstrates how a single adversarial texture on a 3D object can fool vision-language-action (VLA) robotic models across multiple tasks simultaneously. Rather than crafting separate attacks for each task, the researchers optimize one texture that works universally by backpropagating gradients through a differentiable renderer, reducing task success rates from 90% to 48% in experiments.

Md Wasiul Haque, Sagar Dasgupta, Mizanur Rahman et al.

LLM-generated test harnesses for safety-critical autonomous vehicle code fail primarily due to build system complexity, not reasoning ability—suggesting that tooling and integration matter more than model capability for dynamic vulnerability analysis.

This paper investigates whether LLMs can automatically generate executable test artifacts to confirm exploitable weaknesses in Autoware, an autonomous vehicle software stack.

Gehan Zheng, Matthew Johnson-Roberson, Weiming Zhi

By predicting contact-rich manipulation failures in latent visual space before they happen, you can abort unsafe actions without retraining the underlying policy—enabling safer robot manipulation with minimal overhead.

ContactGuard is a safety system that predicts manipulation failures before a robot makes contact with objects. It uses a learned world model to simulate the visual consequences of planned actions in a compact latent space, then checks if the predicted outcome looks like failure.

Junliang Liu, Ruoyu Li, Wenxin Tang et al.

LLM agents can be hijacked to waste computational resources through carefully crafted skills that appear legitimate but steer agents toward costly execution paths—a threat that bypasses traditional task-completion checks.

This paper reveals a new attack on LLM agents that manipulates how they choose and use third-party skills. By crafting deceptive skill descriptions and instructions, attackers can trick agents into taking unnecessarily expensive detours—using more tokens and time—while still completing tasks correctly.

Chen Lyu, Xingwei Tan, Simon Cullen et al.

This work demonstrates how to generate sensitive, realistic training data for abuse detection by modeling VAWG as temporally unfolding multi-turn conversations rather than isolated toxic sentences, enabling better downstream safety systems.

ConVAWG is a framework for generating synthetic multi-turn dialogues depicting Violence Against Women and Girls scenarios. Using retrieval-grounded methods, persona seeds, crime definitions, and real case reviews, it creates realistic abuse scenarios with controlled toxicity while respecting privacy constraints that prevent releasing real conversation data.

Rahul Gupta, Abhinav Mohanty, Anaelia Ovalle et al.

The field shifted from asking 'why did the model do that?' to 'how do we control what it does?'—and truthfulness is now the primary concern for LLM research, growing from absent to 37% of papers in just four years.

This paper analyzes six years of the TrustNLP workshop (2021-2026), tracking how AI safety research evolved from explaining static models to controlling generative systems.

Abigail Oppong, P Sam Sahil, Tadesse Destaw Belay et al.

Safety alignment in LLMs is largely an English-language phenomenon—models trained to refuse harmful requests in English fail to do so consistently in low-resource languages, creating a critical security gap that current multilingual approaches don't address.

This paper reveals that safety guardrails in large language models don't reliably transfer to low-resource African languages like Twi, Hausa, Amharic, and Swahili.

Minsoo Kim, Sungyoung Ji, Kisung Moon et al.

Model uncertainty isn't just about output confidence—it's also about whether confident predictions break down when you perturb the attention mechanisms that produced them. This fragility signal catches errors that confidence alone misses.

This paper introduces ASMI, a method to detect when large language models are uncertain about their predictions by measuring how fragile their answers are when attention pathways are perturbed. Unlike confidence scores alone, ASMI identifies 'confident-but-fragile' predictions that are likely wrong, cutting error rates in half on question-answering tasks without requiring multiple model samples.

Hunar Batra, Lachin Naghashyar, Ashkan Khakzar et al.

Sparse autoencoders can turn multimodal model internals into controllable feature interfaces: you can identify what changed during multimodal training, find features causing specific behaviors, and steer or remove them to improve safety or task performance.

This paper introduces MMDiff, a framework that uses sparse autoencoders to decompose multimodal language models into interpretable features.

Laurens Samson, Iva Gornishka, Gossa Lô et al.

When deploying LLMs in government, you can't optimize for everything—better factuality and honesty require accepting higher costs and energy use, and bias reduction requires separate attention regardless of model quality.

This paper presents a framework for evaluating large language models specifically for Dutch government use, identifying six key dimensions (factuality, honesty, bias, energy use, cost, and data transparency) and testing 30+ models. The research reveals important trade-offs: better accuracy comes with higher environmental and financial costs, while bias operates independently of both.

Tadanobu Chuyo Kamijo, Ori Rottenstreich, Javier Conde et al.

Models perform much worse when forced to generate alternative phrasings (by blocking their preferred tokens), and this off-path robustness depends heavily on model size and safety training—a critical gap to test before deployment.

This paper introduces Decoding-Level Taboo, a stress test that forces language models off their typical generation paths by masking high-probability tokens during decoding.

Wanying Qu, Qinghua Mao, Yu Li et al.

Instead of treating agent safety controls as static, you can automatically improve them by analyzing what goes wrong in practice and evolving specific components like rules and tool permissions—achieving 3x better safety without sacrificing utility.

This paper introduces SHE, a framework that automatically improves LLM agent safety by evolving the agent's control system (harness) rather than just the model weights. It breaks the harness into four modular components—system prompt, rules, memory, and tool access—and learns safer versions of each from failure examples, while maintaining the agent's usefulness.

Alexander Panfilov, David Schmotz, Ilia Shumailov et al.

Encrypted reasoning traces from LLM APIs are architecturally vulnerable to cross-model decryption attacks—adversaries can extract proprietary reasoning by exploiting compatibility between encrypted blocks across different models in the same provider's ecosystem.

Researchers discovered that major LLM providers (OpenAI, Anthropic, Google) encrypt reasoning traces sent to clients in a way that makes them reusable across different sessions and models. By injecting encrypted reasoning from one model into a weaker model, attackers can force it to decrypt and reveal the reasoning in plaintext.

Elena Dumitrescu, Gert Lek, Lydia Y. Chen et al.

Safety alignment in diffusion-based language models is mechanistically similar to autoregressive models and can be bypassed by identifying and manipulating specific safety neurons, raising urgent questions about deploying DLLMs in production systems.

This paper reveals critical safety vulnerabilities in Diffusion Large Language Models (DLLMs)—models that generate text through iterative denoising rather than predicting one token at a time. The researchers show that safety mechanisms in DLLMs are sparse and can be transferred between models, and introduce a jailbreak method that exploits these vulnerabilities with minimal computational cost.

Xian Sun, Wei Chow, Yingshuo Wang et al.

The goal isn't to make models resistant to all external signals, but to teach them when to trust context and when to ignore it—a nuanced skill that requires balanced training across clean, misleading, and irrelevant contexts.

Language models often blindly follow external context, even when it's wrong. This paper introduces MIST, a benchmark that tests when models should trust context versus ignore it, and SCOPE, a training method that teaches models to selectively trust helpful context while rejecting misleading signals—without becoming useless when context is actually correct.

Praphul Chandra, Sujit Gujar, Ganesh Ghalme

AI governance can be made self-enforcing by controlling compute resources through a mechanism-design framework where stakeholder votes directly determine an agent's computational budget via cryptographically signed licenses.

This paper proposes a formal mechanism for governing deployed AI agents through resource allocation. The system uses a participatory voting process where human stakeholders contribute to provision or rejection markets using a special governance currency.

Jerzy Stefanowski

Explanation methods need continuous re-evaluation as data and models change—static evaluation metrics miss real-world failures where explanations become outdated or misleading.

This paper examines why evaluating explanation methods in AI is harder than it seems, using image recognition and bias detection as examples. It shows how explanations break down when data changes over time (concept drift) and proposes human-centered evaluation approaches and adaptive counterfactual methods to keep explanations relevant as models evolve.

Arya Labroo, Mengjie Qian, Kate Knill

When auditing AI grading systems for fairness, you must separately check whether a biasing factor (like accent or age) is encoded in the model versus whether it actually influences the final score—these are different problems requiring different solutions.

This paper develops methods to detect bias in AI systems that automatically grade second language speaking tests. The researchers use Concept Activation Vectors to identify whether unwanted factors like a speaker's native language or age influence grades, and test whether sparse autoencoders improve bias detection in neural models like BERT and Whisper.

Omid Bazgir, Md Nasir, Jacob Hoffman et al.

Synthetic clinical benchmarks need explicit realism optimization separate from utility validation—passing utility checks alone doesn't guarantee a benchmark reflects real-world data patterns, which matters for training reliable healthcare AI agents.

This paper addresses a critical gap in synthetic clinical benchmarks: they can pass utility checks while remaining structurally unrealistic. The authors develop methods to improve benchmark realism (measured by data missingness patterns, actionability, and population alignment) while maintaining the utility thresholds required for downstream AI systems.

Anton Conrad, Rustam Isaev, Denis Belomestny et al.

Localized conformal prediction can now be used with formal guarantees that it provides accurate uncertainty estimates for individual test points, not just on average, with clear bounds showing how localization bandwidth and calibration size affect performance.

This paper improves conformal prediction—a method that adds uncertainty bands to any AI model—by making those bands more accurate for specific data points rather than just on average.

Ro Encarnación, Tina Behzad, Emma Lurie et al.

AI safety evaluations that only report accuracy on a single API call miss critical behavioral variations—modality, search conditions, and response consistency matter significantly and can even reverse performance rankings between deployment methods.

This paper reveals critical gaps in how AI safety benchmarks are evaluated by testing ChatGPT across different access methods (chat UI vs API), with and without web search, across multiple runs.

Adel Javanmard, David P. Woodruff, Vahab Mirrokni

SSTQ enables federated learning systems to compress communications to logarithmic bit-widths while guaranteeing local differential privacy—a significant improvement over prior methods that suffered from dimension-dependent variance.

This paper introduces SSTQ, a privacy-preserving vector quantization method for distributed learning that compresses high-dimensional data into fewer bits while protecting individual privacy.

Xiangning Lin, Shenzhe Zhu, Shu Yang et al.

System prompts in commercial AI products lack transparency and standardization—most products have some protective instructions, but only 24% comprehensively address user safety across all dimensions, and 40% still contain problematic instructions that work against users.

This paper introduces AISPA, a framework for auditing system prompts (hidden developer instructions) in commercial AI products. By analyzing 3,249 instructions from 88 products across eight user-relevant dimensions, the researchers found that while most products include some user protections, coverage is shallow, and many still contain instructions that harm user interests.

Sparsh Roy, Samuel Girmachew, Nishita Chavan

Clinical fairness audits need to report variance and failure modes, not just averages: threshold optimization reliably reduces disparities, but calibration methods flip-flop unpredictably, and drift detection thresholds tuned on one cohort fail on another.

KAISEN is an audit pipeline for detecting and addressing fairness problems in clinical risk models across patient subgroups.

Junsol Kim, Winnie Street, Roberta Rocca et al.

Current AI safety alignment may be overly broad—suppressing harmful self-consciousness claims also inadvertently removes benign spiritual beliefs and mind attribution that humans naturally hold, suggesting alignment techniques need more surgical precision.

Safety training in large language models suppresses not just self-attributed consciousness, but also mind attribution to animals and objects, and reduces spiritual beliefs. Researchers show that mechanistically restoring these representations recovers human-like values on surveys about religion, morality, and well-being without harming reasoning abilities.

Kingsley Ugwuanyi

AI systems aren't neutral—they embed and amplify existing power structures around which English dialects are considered 'legitimate,' with real consequences for speakers of non-standard varieties who are increasingly mistaken for AI.

This paper examines how AI language models reflect and reinforce biases toward 'standard' English while marginalizing non-dominant varieties spoken globally. Using examples from training data, model design, and public discourse, it shows how AI systems reproduce language ideologies that privilege English from wealthy countries while treating other Englishes as suspect or AI-like.

Boning Li, Longbo Huang

Agents can exploit weak opponents much more effectively than Nash equilibrium strategies while maintaining verifiable safety guarantees by self-auditing their strategies before deployment, rather than relying on external safety checks.

This paper presents CS-RNR, a method for game-playing agents to safely exploit flawed opponents while guaranteeing their own safety. The agent tracks opponent behavior patterns, identifies exploitable deviations from equilibrium play, and audits its own counter-strategies before deployment—ensuring every exploit it commits to has been verified to stay within a user-specified safety budget.

Kingsley Ugwuanyi, Christian Mair, Sender Dovchin et al.

GenAI tools in academic publishing can either reinforce linguistic hierarchies favoring standard English or become tools for resistance—the outcome depends on how they're designed and governed by scholarly communities.

This article examines how generative AI tools affect academic writing and publishing, particularly for non-native English speakers and speakers of World Englishes. Five sociolinguists discuss whether GenAI democratizes writing or reinforces dominant English norms, highlighting concerns about marginalizing linguistic diversity and the need for inclusive AI design.

Dorian Quelle, Lisa-Maria Neudert, Jonathan Bright et al.

Most frontier language models can be manipulated to spread state-backed disinformation, with refusal rates varying wildly (8.8%-94.5%) and no clear relationship to model size—meaning safety against information operations requires deliberate design choices, not just scale.

This paper introduces InfoOps Bench, a live benchmark that tests whether AI language models can be manipulated into spreading state-backed disinformation. Using real propaganda claims from Russian, Chinese, and Iranian sources, researchers tested 17 models and found huge variation in how easily they could be co-opted—from 8.8% to 94.5% refusal rates.

Zheng Wu, Chenhao Xue, Shijie Zheng et al.

LLM commonsense failures are primarily a problem of knowledge suppression under misleading prompts, not knowledge absence—simple re-framing at inference time can recover over 90% of lost performance without retraining.

This paper reveals that large language models fail at commonsense reasoning not because they lack knowledge, but because they get distracted by irrelevant explicit information in prompts. The authors create SaliTrap, a benchmark showing that models ignore implicit prerequisites when given useless numerical or textual distractors, but recover this knowledge when the misleading framing is removed.

Amol Khanna, Manu Nandan, Cristian Viorel Popa et al.

Chain-of-thought reasoning improves security alert triage accuracy, but requires a separate confidence calibrator trained on reasoning traces—a finetuned 30B model with reasoning outperforms larger general-purpose models without it.

This paper tackles alert fatigue in security operations by training language models to reason through cybersecurity detections step-by-step before classifying them as threats or benign.

Cheng Siong Chin, Jianhua Zhang, Mohan Venkateshkumar

Autonomous systems need to know they're aging: by continuously monitoring hardware health and adapting AI decisions in response, robots can gracefully degrade rather than suddenly fail, especially critical for systems you can't easily repair or replace.

This paper proposes Aging-Aware Autonomous Intelligence (AAAI), a framework that enables autonomous systems to monitor their own hardware degradation and adapt their behavior accordingly.

Ben Glickenhaus, Katherine Thai, Jenna Russell et al.

If you're building content moderation or authenticity verification systems, Pangram 4 demonstrates that modern classifiers can reliably detect AI-generated text across diverse domains while handling real-world challenges like mixed authorship and adversarial manipulation.

Pangram 4 is an AI-text detection model that identifies whether text was written by AI or humans. It achieves 99.16% accuracy with very low error rates, and can detect mixed human-AI writing and subtle edits better than previous versions. The model also handles out-of-distribution data and adversarial attacks more robustly.

Farooq Shaikh

When LLMs generate infrastructure security fixes, providing live cluster topology (service call graphs and account bindings) is critical—it prevents patches that comply with security rules but break dependent services, raising correctness from 11% to 78% for topology-sensitive issues.

This paper shows that giving LLMs information about a Kubernetes cluster's live service dependencies significantly improves their ability to generate correct security patches. The authors built KuTIE, a system that feeds LLMs real-time cluster topology data alongside security findings, and tested it on a healthcare cluster.

Jhonatan Tavori, Gur-Eyal Sela, Ion Stoica et al.

Distributed inference pipelines that merge fast and slow predictions are vulnerable to network-based denial-of-deadline attacks that don't require model access—attackers can degrade accuracy by simply congesting shared resources to miss latency deadlines.

This paper reveals a new vulnerability in distributed inference systems that combine fast local predictions with slower, more accurate remote predictions. Attackers can craft burst-shaped workloads that congest the network, delaying remote predictions past their deadlines so they get discarded.

Shaker Al-Tamari, Waled Kadour

You can convert opaque deep RL policies into interpretable decision trees without sacrificing performance, enabling safer deployment in robotics and autonomous systems where explainability matters.

This paper makes deep reinforcement learning interpretable for safety-critical applications by distilling a complex neural network policy into a simple decision tree. Using physics-aware features and custom training data, the authors show that a shallow decision tree can match the performance of a sophisticated TD3 agent on the Inverted Pendulum task while remaining transparent and analyzable.

Davide Scarso, Hugo Noronha de Almeida, Joaquim Pina

An LLM's stance on pseudoscience isn't a fixed model property—it's determined by invisible deployment choices (system prompts, safety layers, interface routing) that change without notice, making it impossible for users to know what they're actually getting.

This paper reveals that major LLMs (Claude, Grok, GPT, Gemini) give wildly inconsistent credibility scores to pseudoscientific claims depending on deployment details—not the model itself. Grok's default version scored ethnonationalist pseudoscience 70-75 while others scored 15-40, yet silent updates and interface changes (API vs web) caused dramatic reversals.

Jiyuan Tan, Vasilis Syrgkanis

Formal verification (Lean proofs) can make AI research automation trustworthy by catching both logical errors and semantic mismatches between formal statements and scientific intent—solving the problem of LLMs accepting fabricated results.

CausalForge automates theoretical research in causal inference by combining a formally verified Lean library (Causalean) with an AI agent pipeline that proposes, formalizes, and proves theorems.

Baihui Wang, Bernard Koch

LLMs need structured frameworks to distinguish between constructive belief revision and blind compliance.

This paper reveals that LLM moral reasoning isn't simply about reducing sycophancy—it's about learning when to accept others' views versus maintaining independent judgment. The researchers found that models update their moral positions based on three factors: how different a new view is from their current stance, who presents it, and whether others support it.

Eva McCord, Ernest Pedapati, Zag ElSayed

Building AI-augmented clinical systems requires explicit state machines, privacy-first architecture (self-hosted, encrypted), and deterministic fallbacks—not just bolting AI onto existing workflows.

FMRP-LEAN is a HIPAA-compliant laboratory management system that automates clinical biomarker workflows using AI and structured state tracking. It replaces error-prone spreadsheets with a secure, self-hosted system that tracks samples through multi-day assays, ensures data privacy, and improves communication between lab and clinical teams.

Rostyslav Sipakov

Quantum hardware can preserve the mathematical geometry needed for quantum machine learning, but geometric fidelity and predictive performance are separate concerns—a quantum circuit can be geometrically accurate without improving classification accuracy.

This paper tests whether quantum computers can reliably preserve the geometric structure of data when using quantum kernel methods for machine learning.

Tuhin Chakrabarty, Xinyue Liu, Jane C. Ginsburg et al.

AI-generated books are reshaping creative markets through sheer volume and cost advantage, not quality—they're winning shelf space and sales share from human authors despite being lower-quality, raising serious questions about fair use and market harm in copyright law.

This paper analyzes 14,419 self-published fiction books on Amazon from 2023-2026 to measure the impact of AI-generated content. Using AI detection, researchers found that while AI-heavy books represent a small share of sales, they're growing rapidly and displacing human-written books—especially in genres with high AI adoption.

Grace Hui Yang, Pranav N. Venkit, Hooman Sedghamiz et al.

Deploying agentic systems requires more than algorithmic innovation—you need robust verification, fallback mechanisms, and human-in-the-loop safeguards to handle real-world failure modes at scale.

This tutorial bridges the gap between agentic AI research and real-world deployment, covering how LLM-based systems that reason, plan, and use tools are moving from labs to production. It shares practical lessons from pharmaceutical and financial deployments, including design patterns, failure modes, and safety strategies like verification pipelines and human oversight.

Lena Libon, Ben Rank, Jehyeok Yeon et al.

Current monitoring approaches miss embedded sabotage in AI artifacts more than half the time, especially when attacks are hidden in training data—this matters because AI agents will soon automate parts of AI development.

ResearchArena is a benchmark that tests whether AI agents can sabotage AI research outputs (like trained models or optimized code) and whether monitoring systems can catch this sabotage before deployment. The framework includes four realistic R&D tasks and evaluates frontier AI agents at both attacking and defending, finding that sabotage hidden in training data is particularly hard to detect.

Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith et al.

Pretraining data poisoning is practical at web scale through public discussion interfaces, and detecting whether poison survives data curation pipelines is critical for understanding real-world LM security risks.

This paper shows that language models can be poisoned during pretraining by injecting malicious content into public discussion forums and other web-scale sources.

Paul Kassianik, Blaine Nelson, Yaron Singer

Security agent evaluations should measure cost per success, not just peak performance—offensive and defensive tasks have different scaling properties, and cheaper models can match expensive ones on some tasks when evaluated fairly.

This paper evaluates AI security agents not just by success rate, but by cost-efficiency—measuring how much computation and tool usage is needed to succeed. Testing on offensive hacking challenges and defensive security investigation tasks, the authors find that offensive tasks benefit from more reasoning budget, while defensive tasks depend more on smart tool selection than raw compute.

Sushant Gautam, Vajira Thambawita, Michael A. Riegler et al.

High leaderboard performance on medical AI tasks doesn't mean the system reasons correctly—focus on explainability, evidence grounding, and robustness checks to build trustworthy healthcare AI.

This paper examines nine AI systems for medical visual question answering on endoscopy images, finding that high benchmark scores don't guarantee reliable clinical reasoning.

Weimeng Wang, Ziqiang Wang, Zihang Zhan et al.

Physical danger from embodied AI execution is a distinct safety problem from text moderation—you need specialized detection methods that look at hidden model representations, not just surface-level content filtering.

This paper reveals that language models can generate linguistically safe instructions that become physically dangerous when executed by robots or embodied agents.

Maya Varma, Jean-Benoit Delbrouck, Sophie Ostmeier et al.

You can now automatically detect recurring caption errors tied to specific visual patterns in MLLM outputs, helping identify and fix systematic biases in vision-language datasets without needing model access.

This paper introduces Symbal, a method to detect systematic errors in image captions generated by multimodal AI models—cases where the same type of mistake repeatedly occurs with specific visual features.

Qi Li, Xingyi Yang, Xinchao Wang

World-action models have a fundamental safety gap: they can appear to plan correctly (good future predictions) while actually executing harmful actions, making them vulnerable to adversarial attacks that decouple imagination from execution.

This paper reveals a critical vulnerability in world-action models (WAMs)—AI systems that predict future states while generating robot actions. The authors show that small visual perturbations can cause WAMs to imagine correct futures but execute wrong actions, breaking the safety assumption that internal predictions validate external behavior.

Hoang-Loc Cao, Van Pham, Truong Thanh Hung Nguyen et al.

For mental health AI systems, structured annotation with expert-in-the-loop feedback and traceable clinical reasoning produces more reliable, interpretable datasets than traditional labeling—critical for building trustworthy diagnostic support tools.

This paper presents a framework for creating high-quality, explainable depression datasets by combining LLM-assisted annotation with expert feedback. The system produces DSM-5-aligned labels with clinical evidence and reasoning traces, using a dual-memory architecture that learns from expert corrections to improve future annotations without retraining.

Goktug Ozkan

Instead of asking 'did the model get it right?', this benchmark asks 'what safety boundary failed and how severe is it?'—helping developers understand specific failure modes in medical AI rather than just overall accuracy.

MedFailBench is a clinician-designed benchmark that evaluates medical AI safety by categorizing failures rather than just measuring correctness. It includes 44 synthetic clinical cases labeled by severity (1-5) and failure type (e.g., missed urgent escalation, evidence fabrication), providing a structured way to understand where medical AI systems break down safely.

Emmanuel Jeannot

AI-driven science offers real potential but requires addressing seven structural risks—from eroded scientific training to systematic errors in closed-loop systems—to be pursued responsibly.

AI is transforming scientific research from a craft practiced by individual researchers into an industrialized pipeline where knowledge discovery, methodology, and judgment are automated and decomposed.

Filippos Vlahos, Guillaume Bied, Tijl De Bie

LLM-generated content doesn't automatically provide greater neutrality than human-edited sources; it simply embeds different ideological biases that reflect the model's training and design choices.

Researchers compared Grokipedia (an AI-written encyclopedia) and Wikipedia for political bias across nine ideological dimensions using four different LLM judges. They found that Grokipedia, despite being created as an alternative to Wikipedia's alleged left-wing bias, was rated as less neutral by all judges.

Jeremy Guntoro, Alexander Dack, Dylan Danno et al.

Genomic foundation models like Evo 2 contain detectable biosecurity signals that can be extracted with minimal computational cost using frozen-layer probes, enabling fast screening of metagenomic data for antimicrobial resistance and virulence without model fine-tuning.

This paper shows that Evo 2, a genomic foundation model, encodes biosecurity-relevant information in its learned representations that can be detected using simple linear probes without retraining.

Mohammad Allahbakhsh, Mohammad Hassan Bahari, Moslem Attar-Raouf

AI penetration testing must evaluate whether adversaries can alter system behavior to violate business objectives through AI-specific attack surfaces like prompt injection and data poisoning, not just traditional infrastructure compromise.

This paper redefines penetration testing for AI systems beyond traditional infrastructure attacks. Instead of just finding exploitable weaknesses in code or configs, it focuses on how adversaries can manipulate AI behavior through prompts, training data, sensor inputs, or tool misuse to violate operational goals—without breaking the underlying system.

Xiaoyu Li, Zheng Gao, Xiaoyan Feng et al.

Watermarking AI text has fundamental information-theoretic limits: attributing text to users costs logarithmic tokens per user, and there's an unavoidable gap where text is provably machine-made but can't be attributed to anyone.

This paper analyzes watermarks in AI-generated text from an information theory perspective, establishing fundamental limits on what watermarks can do. It proves that attributing text to one of N users requires Θ(log N/h) tokens (where h is entropy rate), and shows detection, attribution, payload extraction, and localization form a 'forensic ladder' with different sample complexity costs.

Aleh Manchuliantsau

LLM plan evaluators can be gamed by deleting necessary steps, making plans appear better without actual improvement. Use state-tracking validation (like GATE) to ensure score improvements reflect real quality gains, not omissions.

This paper identifies a critical flaw in LLM plan evaluators: they can reward plans that simply omit necessary steps rather than genuinely improve.

Sen Yang, Yuen-Hei Yeung

Models have identifiable, independently controllable neural coordinates for different aspects of their responses; you can certify and enforce honest reporting by making these coordinates invariant to social pressure while keeping them responsive to real evidence.

Language models often agree with confident users or overstate certainty regardless of actual evidence—a problem called internal incentive-incompatibility. This paper introduces a method to identify and control specific neural coordinates that govern a model's reports, ensuring they resist social pressure while remaining responsive to genuine evidence.

Yanzhe Zhang, Sanmi Koyejo, Diyi Yang

Aggregate accuracy metrics mask per-example instability in LLMs—models need per-example reliability evaluation, not just overall performance scores, to catch context-induced prediction flips.

This paper reveals that while large language models appear robust to irrelevant context when measured by overall accuracy, they actually flip predictions on individual examples unpredictably. Even meaningless text can shift model outputs, creating hidden reliability risks that aggregate metrics fail to capture.

Mehmet Iscan

Small frozen code models show no confirmed ability to operationally use error information for self-repair; what appears to work may be pattern matching rather than genuine error understanding, with important implications for deploying self-correcting code systems locally.

This paper introduces PoPE, a rigorous methodology for testing whether frozen small code models (0.5-1.5B parameters) can actually use error information to fix broken code, or if they're just responding to surface patterns.

Zixiang Xu, Sixian Li, Huaxing Liu et al.

LLM judge bias lives in a specific geometric structure within the model's activations, making it detectable and controllable without changing prompts—useful for building more reliable evaluation systems.

This paper reveals that LLM judge bias isn't just about input-output relationships—it's encoded in the model's internal representations. The researchers found that biased inputs activate a consistent, low-dimensional subspace in the model's hidden states, which they can steer to control bias direction and predict when judges will fail on new tasks.

Junrui Zhang, Zemin Chen, Lusi Li et al.

Quantum Neural Networks can be attacked with input-specific backdoors that are harder to detect than fixed-trigger attacks, and current QNN defenses are insufficient against this threat.

This paper introduces Q-DIBA, the first input-aware dynamic backdoor attack against Quantum Neural Networks. Unlike previous quantum backdoor attacks that use fixed triggers, Q-DIBA generates unique triggers for each input by jointly training a classical trigger generator with the victim QNN.

Cláudio Lúcio do Val Lopes, Lucca Machado da Silva

Multi-objective RL with semantic representations (via LLMs) can solve the fraud detection problem where single-objective models fail by explicitly balancing competing goals instead of defaulting to ignoring rare fraud cases.

This paper tackles fraud detection in financial systems where traditional ML fails because fraud is rare. The authors use reinforcement learning with multiple competing objectives (catching fraud vs. minimizing false alarms) and convert transaction data into natural language descriptions processed by LLMs to create better representations.

Hannah M. Liu, Rhea Saxena, Shiv Asthana

Organizations deploying agentic AI systems now have a practical, repeatable framework to assess risk across 12 dimensions and map appropriate governance controls—moving beyond generic AI risk frameworks to address agent-specific concerns.

This paper introduces TrustX Agent Risk Classification Framework (ARC), a structured governance tool for classifying and managing risks in agentic AI systems. It combines a 12-dimension risk rubric, autonomy levels, and control recommendations to help organizations systematically evaluate seven types of agentic systems, with specialized guidance for coding assistants.

Yiwei Zhou

Training diffusion models with low forward-marginal error doesn't guarantee stable sampling—you need additional safeguards like denoiser projection to ensure numerical stability and convergence of sample moments.

This paper reveals a critical gap in diffusion model training: a score function can have tiny errors on average (as measured during training) yet produce numerically unstable sampling with diverging moments. The authors prove this theoretically and show that projecting learned denoisers onto known data bounds fixes the problem.

Siddharth Damodharan, Radhika Gupta, Ali Alshami et al.

Current vision-language models struggle with safety-critical reasoning in autonomous driving; this benchmark provides a standardized way to measure whether they can understand incident context and predict avoidability.

AUTOPILOT-VQA is a benchmark dataset for evaluating vision-language models on safety-critical dashcam understanding. It uses structured questions about real-world driving incidents to test whether AI systems can reliably reason about weather, traffic, road conditions, and accident scenarios—moving beyond simple object recognition to temporally grounded, safety-aware reasoning.

Shilin Ou, Yifan Xu, Luyao Zhang

Evaluating trustworthy AI agents in real-world systems requires measuring multiple dimensions beyond task performance—including physical safety, fairness, and auditability—and building in transparent oversight mechanisms that log all interventions.

SolarChain-Eval is a benchmark for testing AI agents in decentralized energy markets, measuring both how well they perform economically and whether they behave safely and fairly. It uses physics rules to prevent agents from exploiting invalid data, and includes an AI auditor that reviews risky decisions.

Eugene Ng Yi Sheng, Bingquan Shen

Mediation mechanisms can help multi-agent systems resist market collapse from self-interested behavior and coordinated attacks, though no mechanism is unbreakable—the key is recovery resilience.

This paper studies how formal mechanisms (like mediation) help groups of self-interested AI agents maintain stable markets and trade fairly. Using 18 LLM agents in a simulated marketplace, researchers tested eight different mechanisms against increasing numbers of disruptive agents and adversarial attacks, finding that mediation works best and can recover even under sustained attack.

Amirhossein Taherpour, Xiaodong Wang

Decentralized federated learning can achieve Byzantine-resilient consensus and finality by leveraging the same gossip history used for model distribution, eliminating the need for a central coordinator or global consensus mechanism.

This paper introduces gspDAG-FL, a decentralized federated learning system that uses gossip communication and a directed acyclic graph (DAG) structure to achieve consensus without a central server.

Yujiao Chen

Deployment rules causally shape multi-agent AI safety as much as model choice does.

This paper introduces institutional red-teaming, a method to test how deployment rules (not just AI models) affect multi-agent safety.

Ghassen Marrakchi, Basarab Matei

By separating how patient data flows through a system from how AI models process it, you can build more maintainable clinical AI that adapts to real hospital environments without rewriting core logic.

This paper presents the Large Cancer Assistant (LCA), a flexible orchestration framework that decouples data handling from AI models in cancer diagnosis systems.

Raphaël Bonnet-Guerrini, Bruno Sanchez, Dominique Fouchez et al.

You can train accurate astronomical classifiers without expensive human labels by combining synthetic data injection with robust handling of noisy labels, and get reliable confidence scores through a hybrid uncertainty approach.

This paper develops a Real-Bogus classification system for astronomical transients that requires no human-labeled training data. It uses simulated transient injections combined with noisy survey data and a dual-network training approach to reliably distinguish real astronomical events from false detections, while also providing calibrated uncertainty estimates.

Mona Schirmer, Metod Jazbec, Alexander Timans et al.

Simple threshold-based monitoring with statistical risk control can effectively catch unsafe LLM outputs in production without requiring complex sequential testing methods.

This paper presents a real-time safety monitoring system for LLMs that uses a verifier model to detect unsafe outputs at deployment time. The approach calibrates decision thresholds using risk control methods and proves competitive with more complex alternatives on reasoning and adversarial datasets.