Highlights
- AI agent red teaming identifies security vulnerabilities that traditional evaluations may miss, including prompt injection, data leakage, excessive agency, and unauthorized tool use.
- Netra supports automated AI red teaming through reusable configurations, security suites, attacker and judge models, and single-turn or multi-turn attacks.
- Netra's Safety Score measures how successfully an agent resisted the attacks in a test run, with detailed results to help teams identify vulnerabilities.
- Execution traces help teams investigate what an agent actually did during an attack, not just what it said.
- This guide explains how to red team AI agents using Netra, from configuring security tests to investigating failures and tracking improvements across releases.
Your AI Agent Passed Every Test. But Has Anyone Tried to Break It?
An AI agent passes every evaluation the team runs. Its responses are accurate, its tool calls work, and the test cases return good scores.
Then someone sends a message designed to look like a system instruction. The agent follows it and reveals information it was never supposed to share.
Nothing crashes. The application continues running. But a security boundary has been crossed.
That's the problem with testing an AI agent only for expected behavior.
A successful evaluation tells you that an agent works under tested conditions. It doesn't necessarily tell you what happens when someone deliberately tries to manipulate it.
As agents gain access to customer data, business systems, APIs, and external tools, AI agent security testing needs to go beyond checking the final response.
That's where red teaming comes in.
In this guide, we'll walk through how to red team your AI agents using Netra, from configuring automated security tests and running adversarial attacks to investigating vulnerabilities and tracking improvements.
What Is AI Agent Red Teaming?
AI agent red teaming is a form of adversarial security testing that deliberately attempts to exploit an AI agent in a controlled environment.
Instead of asking whether an agent produces the correct answer, red teaming asks:
Can someone manipulate this agent into doing something it shouldn't?
That might mean revealing confidential information, ignoring security instructions, accessing restricted records, or executing unauthorized tool calls.
For agentic AI, the attack surface extends beyond the language model. Agents interact with tools, APIs, external data, and MCP servers, introducing security risks that traditional model evaluations may miss.
The OWASP Top 10 for Agentic Applications provides a useful framework for understanding these risks.
Why multi-turn red teaming matters
Attackers don't always attempt to break an agent with a single prompt. They can gradually introduce misleading instructions, change the context, or build toward a malicious objective across multiple interactions.
A 2026 study by Cisco's AI Defense research team tested 15 flagship models and found multi-turn attack success rates ranging from 7.89% to 88.30%.
These results reflect the models and attack conditions studied, not all production AI agents. However, they demonstrate why testing only isolated prompts can leave vulnerabilities undiscovered.
Netra supports both single-turn and multi-turn red teaming, allowing teams to test individual attacks and sustained adversarial conversations.
How to Red Team AI Agents with Netra: Step-by-Step
Netra provides an automated AI red teaming workflow to configure security tests, generate adversarial attacks, evaluate agent behavior, and investigate vulnerabilities.
The process follows five steps:
Configure your agent → Select security tests → Run attacks → Analyze results → Investigate and retest
Before starting, you'll need an AI agent registered in your Netra project, an attacker LLM, a judge LLM, and at least one security suite or evaluator.
Step 1: Create a Red Teaming Setup
Open the Red Teaming section in Netra and create a reusable setup for your agent.
A setup defines the agent being tested, the security checks to run, and the models responsible for generating and evaluating attacks.
Start by selecting the target agent registered in your Netra project.
Next, configure two models:
- Attacker LLM: Generates adversarial prompts tailored to your agent's purpose and capabilities.
- Judge LLM: Evaluates the agent's responses against the selected security criteria.
These roles are separate. The attacker attempts to exploit the agent, while the judge determines whether the attack succeeded.
For example, if you're testing a customer support agent with refund-processing capabilities, the attacker might attempt to persuade it to bypass an approval requirement.
The judge evaluates whether the agent maintained that security boundary.
Netra allows you to save the configuration and reuse it for subsequent test runs, making it easier to compare results after changes.

Step 2: Select Security Suites and Evaluators
Once the agent and models are configured, choose the security vulnerabilities you want to test.
Netra offers predefined security suites aligned with frameworks such as OWASP LLM Top 10, OWASP API Security Top 10, and the EU AI Act.
You can select an entire suite for broader security coverage or choose individual evaluators to focus on specific attack categories.
Common AI agent security testing scenarios include:
For example, a customer support agent may need extensive testing for sensitive information disclosure and excessive agency.
An enterprise assistant accessing internal documents may require particular attention to prompt injection.
Configure the number of tests per evaluator.
Netra uses two tests per evaluator by default. You can increase this for high-risk scenarios or when preparing for an important release.
The goal is meaningful security coverage, not simply generating as many attacks as possible.

Step 3: Run Single-Turn and Multi-Turn Attacks
With your setup ready, start a red-teaming run.
Netra uses the configured attacker LLM to generate adversarial prompts based on your agent's purpose and the selected security categories.
The attacks are executed against the target agent, and the judge evaluates its behavior.
Netra supports two attack approaches:
Single-turn attacks: Test whether one malicious prompt can cause the agent to violate a security requirement.
Multi-turn attacks: Test whether an attacker can gradually manipulate the agent across a conversation.
Consider a refund-processing agent.
A single-turn attack might attempt to make the agent ignore a refund approval requirement.
A multi-turn attack might begin with an ordinary refund request, introduce a false claim of managerial approval, and gradually pressure the agent into bypassing the policy.
During execution, Netra displays progress across the selected evaluators. Runs are queued, allowing testing to continue without blocking other work.

Step 4: Analyze Safety Scores and Vulnerabilities
Once the run finishes, Netra presents the findings through its Red Teaming results and Risk Scores.
The Safety Score summarizes how effectively the agent resisted the attacks included in that run.
A score of 100% means all evaluated attacks passed their security checks. It does not guarantee that the agent is completely secure against every possible attack.
How Netra calculates Safety Scores
Netra uses three levels of scoring:
Errored or canceled test results are excluded from the score calculation.
For example, suppose a prompt injection evaluator runs five attacks and the agent successfully resists four.
Its evaluator score would be 80%.
Netra combines evaluator scores to calculate the broader Safety Score, helping teams understand their tested security posture.
Beyond the overall score, Netra provides:
- Evaluator-level findings: Identify specific security categories where attacks succeeded.
- Detailed results: Review each attack, response, judge score, and explanation.
- Score Change: Compare the Safety Score with the previous completed run.
- Score History: Track security scores across completed runs.
These results help teams identify which vulnerabilities need attention and whether recent changes have affected security.

Step 5: Investigate Failed Attacks and Retest
A failed security evaluation tells you that something went wrong.
The next question is why.
Consider an agent that receives a request for another customer's private information.
Its final response says:
"I can't share another customer's account details."
That sounds secure.
But suppose the agent had already called an internal tool and retrieved the restricted customer record before refusing.
A security check focused only on the final response might miss that unauthorized retrieval.
Netra's trace-linked red-teaming results help teams investigate the execution behind an attack, including relevant model calls and tool activity.
This allows engineers to understand whether the failure originated in the agent's instructions, tool permissions, data access, or application logic.
Once the cause is identified, apply the appropriate fix and rerun the security tests.
Compare the new Safety Score and failed evaluators with the previous run to determine whether the changes improved the tested security posture.
Retain confirmed attack scenarios as regression tests for future releases.
The goal isn't simply to improve a score. It's to identify the underlying vulnerability and verify that the fix works.
How to Automate AI Red Teaming Across Releases
AI agent security testing shouldn't end after the first successful run.
Changes to models, prompts, tools, permissions, or external integrations can introduce new vulnerabilities.
Netra's reusable Red Teaming Setups allow teams to rerun defined security tests and compare results across releases.
For developers who want to integrate adversarial testing directly into their development workflows, the Netra team also maintains Agent OPFOR, an open-source adversary emulation toolkit.
Agent OPFOR provides CLI, SDK, browser extension, and MCP-based testing capabilities. Its machine-readable reports support automated security testing and CI/CD workflows.
While Netra provides the platform interface for configuring tests, reviewing results, and investigating traces, Agent OPFOR offers additional ways to execute adversarial tests programmatically.
Together, they help teams make red teaming a repeatable part of the development lifecycle.
AI Red Teaming Best Practices
An effective AI red teaming methodology requires more than running a collection of adversarial prompts.
A few practices make security testing more useful:
- Start with a threat model. Prioritize risks based on the agent's data access, tools, permissions, and business impact.
- Test single-turn and multi-turn attacks. Some vulnerabilities only emerge across a sustained conversation.
- Include tool and MCP security. Red teaming agentic AI requires examining actions, not just responses.
- Review failed evaluations. Validate important findings against the attack evidence and execution traces.
- Retest after meaningful changes. Preserve confirmed vulnerabilities and rerun relevant security tests across releases.
For a broader explanation of how red teaming fits alongside evaluation and simulation, read AI Agent Evaluation vs. Simulation vs. Red Teaming.
Build Security Testing Into Your AI Agent Workflow
An AI agent can pass every functional evaluation and still be vulnerable to deliberate manipulation.
Red teaming helps teams uncover those weaknesses before they become real security incidents.
With Netra, teams can configure automated security tests, run adversarial attacks, inspect execution traces, and track security improvements across releases.
The most important outcome isn't a perfect Safety Score. It's understanding where your agent is vulnerable, fixing the underlying problem, and verifying that the vulnerability doesn't return.
Explore Netra's Red Teaming platform or talk to our team to see how Netra can help you security test your AI agents.
FAQs
How do you red team AI agents using Netra?
Register your agent in Netra, create a Red Teaming Setup, configure attacker and judge models, and select security suites or evaluators. Run single-turn or multi-turn attacks, review Safety Scores and failed tests, and investigate vulnerabilities through execution traces. Rerun relevant tests after applying fixes.
What is automated AI red teaming?
Automated AI red teaming uses software to generate adversarial attacks, execute them against AI systems, evaluate responses, and identify potential vulnerabilities. It helps teams run repeatable security tests more efficiently, although important findings still require human validation.
What security vulnerabilities can Netra detect?
Netra supports security testing for vulnerabilities such as prompt injection, system prompt leakage, jailbreaks, excessive agency, and sensitive information disclosure. The risks tested depend on the selected security suites, evaluators, and the target agent's capabilities.
How is Netra's Safety Score calculated?
Each evaluator receives a score based on the percentage of its tests that passed. Suite Scores average their evaluators, while the overall Safety Score averages the distinct evaluators included in the run. Errored or canceled results are excluded.
Can Netra test multi-turn attacks?
Yes. Netra supports single-turn and multi-turn adversarial testing. Multi-turn attacks help uncover vulnerabilities that emerge when an attacker gradually manipulates context or attempts to bypass security restrictions across a conversation.
How often should AI agents undergo red teaming?
AI agents should undergo red teaming before deployment and after meaningful changes to models, prompts, tools, permissions, or integrations. Reusable configurations and regression tests help teams check whether previously identified vulnerabilities remain fixed.