Test across scenarios
Test AI agents across realistic scenarios by combining user goals, account details, and contextual data to see how they respond in different situations..
Test across scenarios: Docs (opens in new tab)Simulation
Test your agent against every kind of user
before the first real one shows up.
Set a goal, pick a persona, and let simulated users hold full conversations with your agent, text or voice, scored automatically.
Trusted by teams building and deploying AI agents
What is Simulation?
Rehearse whole conversations before customers have them. A simulated user, with a goal, a persona and the facts your agent must state, holds a multi-turn conversation with your agent. Evaluators score every run and keep the transcript, so you can see exactly where it went wrong.
A scenario is a user with a reason to be there
Create realistic AI agent testing scenarios with defined user goals, personas, customer details, and expected facts. Test how your agent responds to different situations before real customers encounter them
Test across personas
Simulate neutral, friendly, frustrated, confused, or custom user personas to test how your AI agent handles different communication styles, including de-escalation and clarity..
Test across personas: Docs (opens in new tab)
Create scenarios using AI
Generate AI agent testing scenarios, user goals, and personas based on your agent's purpose, then refine them for realistic simulation runs.
Create scenarios using AI: Docs (opens in new tab)Check the agent says what the policy says
Define the facts and policies your AI agent must communicate, then automatically evaluate whether each simulated conversation conveys them correctly.
Check the agent says what the policy says: Docs (opens in new tab)Catch the failures that only show up on turn five
Single-turn tests pass agents that lose the thread. Simulated users hold whole conversations, so memory, flow and resolution get tested the way customers use them.
Run multi-turn evals for text and voice
Whole conversations with text and voice agents, scored on memory, flow and resolution.
Run multi-turn evals for text and voice: Docs (opens in new tab)
Run against your agent endpoint
Wrap your agent in the BaseTask interface and the simulator drives it turn by turn, with session continuity.
Run against your agent endpoint: Docs (opens in new tab)Run in your CI/CD pipeline
Trigger simulation runs from the Python or TypeScript SDK, so conversations are rehearsed on every build.
Run in your CI/CD pipeline: Docs (opens in new tab)Set how long a conversation runs
Cap the turns per scenario: about five for support, six to eight for technical troubleshooting.
Set how long a conversation runs: Docs (opens in new tab)Score every conversation, automatically
Seven predefined evaluators cover conversation quality and agentic behaviour, each scored from 0 to 1. Add your own for rules only your team knows.
Use a set of predefined evaluators
Five for quality and two agentic, all LLM-as-Judge. A score of 0.6 or higher passes.
Use a set of predefined evaluators: Docs (opens in new tab)Bring evaluators of your choice
Add your own evaluators next to the library when a conversation has rules only your team knows.
Bring evaluators of your choice: Docs (opens in new tab)Know whether the goal was reached
Every scenario has a goal the simulated user is pursuing, so a run ends in a verdict rather than a transcript to read.
Know whether the goal was reached: Docs (opens in new tab)
Get scores per turn and per run
See which turn dropped the score, and how the whole conversation did.
Get scores per turn and per run: Docs (opens in new tab)Debug the turn that went wrong
Every run keeps the conversation, the turn-level scores and the traces behind them, so a failed simulation opens on the turn that failed.
See the detailed conversation view and eval results
The full transcript on one tab and every evaluator's result on another.
See the detailed conversation view and eval results: Docs (opens in new tab)Open the trace behind any turn
Follow a low score down to the spans that produced that reply.
Open the trace behind any turn: Docs (opens in new tab)Confirm the scenario as configured
A third tab shows the goal, persona, data and fact checks the run actually used.
Confirm the scenario as configured: Docs (opens in new tab)Review every run at a glance
Scenarios executed, pass and fail rates, and average cost and latency across all runs.
Review every run at a glance: Docs (opens in new tab)Built for text and voice agents
Drive any agent through BaseTask from Python or TypeScript, whatever framework it runs on, including voice pipelines.
- Python SDK
- TypeScript SDK
- LiveKit
- Vapi
- Deepgram
- ElevenLabs
- Cartesia
- LangGraph
- CrewAI
Use the Python and TypeScript SDKs
Wrap your agent once and run simulations from code or your pipeline.
Docs (opens in new tab)Trace and rehearse voice pipelines
Voice agents built on LiveKit or Vapi, with speech from Deepgram, ElevenLabs or Cartesia, land in the same traces.
Docs (opens in new tab)Cookbook
Rehearse the hard conversations first
Cookbook · Simulation
Simulate customer support conversations
Test customer support AI agents with realistic multi-turn scenarios, user personas, goals, and fact checks. Identify incomplete resolutions and conversation failures that single-turn tests can miss.
Open the cookbook (opens in new tab)What you'll build
- 01Select evaluators for goals, facts, completeness and guidelines
- 02Create a multi-turn scenario with a goal, user data and fact checks
- 03Add scenarios with other personas and intents
- 04Wrap your agent in the BaseTask interface
- 05Trigger the simulation and read transcripts, scores and traces
Frequently Asked Questions
Everything You Need to Know About Simulation
Can't find the answer here? The Simulation docs go deeper, or talk to our team.
What is AI agent simulation?
AI agent simulation is a method of testing AI agents through realistic, multi-turn conversations with simulated users. In Netra, teams define user goals, personas, and expected facts, then run simulated interactions against text or voice agents. Automated evaluators measure conversation quality, memory, factual accuracy, and goal completion, while transcripts and traces help diagnose failures.
How do you test an AI agent using simulation?
To test an AI agent using simulation, define realistic user scenarios, goals, and personas, then run multi-turn conversations against the agent. Evaluate the resulting interactions for accuracy, memory, conversation flow, and goal completion. Netra automates these simulations and provides transcripts, scores, and execution traces to investigate failures.
Can Netra simulate voice agent conversations?
Yes. Netra supports multi-turn simulation for text and voice agents. Teams can test conversation flow, memory, and goal completion while tracing voice pipelines built using supported frameworks and speech providers.
Can AI agent simulations run automatically in CI/CD?
Yes. Netra supports automated simulation runs through Python and TypeScript SDKs, allowing teams to test conversation scenarios as part of their development and CI/CD workflows.
Which personas are available?
Neutral, Friendly, Frustrated, Confused and Custom — from a baseline user to one designed to test de-escalation or clarity, or one you describe yourself.
How are simulations scored?
Seven LLM-as-Judge evaluators — five for quality and two agentic — score each conversation between 0 and 1. A score of 0.6 or higher passes.
How many turns should a scenario have?
Around five turns suits most support conversations; technical troubleshooting usually needs six to eight.
How do I debug a failed simulation?
Open the scenario under Simulation → Test Runs and use its three tabs: Conversation for the transcript, Evaluation Results for the scores, and Scenario Details to confirm it was configured the way you intended.
How does Simulation differ from Evaluation?
Evaluation scores single request-and-response test cases. Simulation scores whole conversations, where the agent has to remember context, handle a persona and reach a goal over several turns.
Start today
Meet your hardest customer in staging
Write a scenario, pick a persona, and read the transcript before anyone real has the conversation. Free to start.