AI security evaluation needs standardized, automated testing frameworks like AVISE to identify vulnerabilities before deployment—the authors show all tested language models can be jailbroken, highlighting the need for systematic security assessment.
AVISE is an open-source framework for systematically testing AI systems for security vulnerabilities. The researchers demonstrate it by creating an automated test suite that discovers jailbreak attacks on language models, finding that all nine tested models are vulnerable to varying degrees.