Training web agents in adversarial simulation with co-evolving curricula and adaptive attackers produces agents that generalize better to real-world prompt injection attacks than agents trained on fixed injections.
This paper presents AdvSim2Real, a training method that improves web agents' robustness against prompt injection attacks. The approach co-evolves three components—a task curriculum, an adaptive adversary, and the agent—within a simulated web environment.