Using learned generative models instead of random mappings for adversarial prompt search produces more realistic attacks that better reflect real-world risks and are harder to filter with input guardrails.
This paper improves adversarial attacks on language models by replacing random prompt generation with a learned diffusion language model. Instead of creating unnatural prompts that are easy to detect, the method generates realistic text while searching for inputs that trigger harmful behaviors like overthinking.