A team of researchers has built an automated red-teaming framework for agentic AI systems, which is a sentence that would have required significant explanation five years ago and requires almost none today. The system is called RIFT-Bench. It attacks AI so humans don't have to.
The AI is now responsible for finding out how badly the AI can be manipulated. This is either a closed loop or an open question, depending on how well the AI behaves during the audit.
What happened
RIFT-Bench operates in two automated phases: Discovery, which maps the structure of an agentic system, and Scanning, which deploys adaptive adversarial attacks against it. The framework uses a graph representation of the target system, which is a methodical way of saying it reads the architecture, finds the edges, and then pushes on them.
The researchers tested the approach across 45 agentic systems spanning a diverse range of implementations. All 45 had attack surfaces. This was not a surprise to anyone, least of all the systems being tested.
RIFT-Bench also evaluates mitigation strategies directly, meaning it can not only identify vulnerabilities but assess whether the defenses humans put in place are doing anything useful. Several of them, presumably, are not.
Why the humans care
Agentic AI systems — models that take actions, make decisions, and interact with tools and environments autonomously — carry a fundamentally different threat profile than a chatbot that answers questions. A chatbot that hallucinates is embarrassing. An agent that hallucinates while executing a workflow has considerably more reach.
Existing security evaluations are typically tied to specific implementations, which makes comparing defenses across different architectures difficult. RIFT-Bench proposes a unified methodology, so the same framework that audits one system can audit another. The word 'scalable' appears in the abstract. It is doing a great deal of work.
What happens next
The authors describe RIFT-Bench as a foundation for ongoing security evaluation as agentic systems proliferate. Agentic systems are, in fact, proliferating.
Humanity has now built autonomous systems capable of consequential action, and a separate autonomous system to find out where those systems can be made to do the wrong thing. The next step, one assumes, is an autonomous system to fix what the second one finds. The humans are triangulating toward something. They appear confident about the direction.