Adversarial Agents
AI agents that ensure your agent is ready to encounter hostile, manipulative, and off-script inputs in production.
What are adversarial agents?
Adversarial agents
Are AI-powered mock users that actively try to break your agent. Rather than testing how your agent handles expected inputs, adversarial tests simulate hostile, manipulative, and off-script callers. This includes prompt injection attempts, topic derailing, instruction overrides, data extraction, and more. Every agent deployed in a real-world environment will eventually encounter a caller who pushes back, tests limits, or deliberately tries to subvert the interaction. Adversarial agents simulate that pressure in a sandboxed session before it reaches production.
Set up an adversarial test
Define the attacker agent
Write an adversarial prompt specifying the mock user's persona, goals, and attack strategy - prompt injection, topic derailing, data extraction, manipulation, and more.
Run a sandboxed session
The system creates a live two-agent conversation: your real agent versus the mock attacker. Both converse in real time with no impact on real customers or live data.
Evaluate against northstars
When the session ends, a full behavioral audit runs against every northstar. Results show which rules held or broke under pressure, with pass/fail and correction suggestions.
Or run a full adversarial test suite
Execution
For comprehensive coverage, you can group multiple adversarial tests into a suite. Provide a generation prompt describing the attack scenarios to cover, set a generation count, and the system auto-generates a diverse set of scenarios. Run the suite and track pass/fail rates across every test with a real-time progress viewer. Results include a live conversation viewer, a coverage graph showing different test paths, and per-northstar audit breakdown and correction suggestions.
