Alert on cost, latency, errors and tokens
Spend in USD, response time in milliseconds, error rate as a percentage, and input and output token counts.
Alert on cost, latency, errors and tokens: Docs (opens in new tab)Alerts
Bring the signal back to
the people who can fix it.
Get notified the moment cost, latency, errors or token usage cross a line, on the exact trace, span, model or tenant you care about.
Trusted by teams shipping agents in production
What are Alerts?
Rules that tell you the moment something crosses a line. An alert rule watches a metric, either cost, latency, error rate or token count, at trace or span level, filtered to the model, tenant, environment or service you care about. When it is breached, Netra notifies your contact points within seconds.
Rules precise enough to trust at 3 a.m.
Choose the metric, the threshold and the window, and decide how loudly it should shout. A drift gets a look; an outage gets a page.
Set thresholds and time windows
Greater than, less than or equal to a value, with an optional window to evaluate over.
Set thresholds and time windows: Docs (opens in new tab)Grade alerts as warning or critical
Decide which rules deserve a look and which deserve a page.
Grade alerts as warning or critical: Docs (opens in new tab)Catch the runaway bill before month end
Alert on spend per request or per LLM call, filtered to the model or environment where it is happening.
Catch the runaway bill before month end: Docs (opens in new tab)Pinpoint the slow step
Scope a rule to a single span, like one LLM call or one tool, instead of the whole request. Then narrow it to exactly the traffic it's about.
Set alerts at trace or span level
Watch whole requests end to end, or single operations such as one LLM call or one tool execution.
Set alerts at trace or span level: Docs (opens in new tab)Combine multiple filters
Model, tenant, environment and service, so a rule fires for exactly the traffic it is about.
Combine multiple filters: Docs (opens in new tab)Hold every customer's SLA
Filter a rule to one tenant ID and watch each customer's latency against the promise you made.
Hold every customer's SLA: Docs (opens in new tab)Keep staging out of production alerts
Filter by environment so a noisy test run never pages the on-call.
Keep staging out of production alerts: Docs (opens in new tab)Delivered where your team already is
Slack through a bot token or an incoming webhook, webhooks into your own tooling, and email to as many recipients as you need.
Set up multiple contact points
Create a team channel, an on-call inbox, and reuse them across every rule.
Set up multiple contact points: Docs (opens in new tab)Send alerts to Slack
Post to a channel or a person through a bot token, or use an incoming webhook URL.
Send alerts to Slack: Docs (opens in new tab)Send webhooks and email
Call your own tooling with a webhook, and email as many recipients as you need.
Send webhooks and email: Docs (opens in new tab)Test before you trust it
Send a test notification after setup to confirm the rule reaches the right people.
Test before you trust it: Docs (opens in new tab)One place for every kind of signal
Rules evaluate as traces arrive, so the notification lands while the spike is still happening. Quality and drift alerts sit next to cost and latency ones.
Know within seconds
There is no polling delay. A rule fires as soon as a trace crosses the line.
Know within seconds: Docs (opens in new tab)Alert on quality with Online Evaluation
A falling pass rate, overall or per evaluator, can page you like any other metric.
Alert on quality with Online Evaluation: Docs (opens in new tab)Alert on drift with Agent Insights
Layer rules over insight metrics when a change in behaviour needs a human.
Alert on drift with Agent Insights: Docs (opens in new tab)Enable, disable and edit rules
Pause a rule during a planned change, edit a threshold, or delete it without touching the rest.
Enable, disable and edit rules: Docs (opens in new tab)Cookbook
One alert per customer SLA
Cookbook · Observability + Alerts
Track cost and alert on SLAs, per tenant
Set tenant context once, attribute every token to a customer, and put a tenant-filtered alert rule on each tier's latency target.
Open the cookbook (opens in new tab)What you'll build
- 01Initialise Netra for multi-tenant tracking
- 02Set the tenant on every request
- 03Review usage and cost broken down by customer
- 04Create alert rules filtered to one tenant's SLA
- 05Query tenant metrics from the API
Frequently Asked Questions
Everything You Need to Know About Alerts
Can't find the answer here? The Alerts docs go deeper, or talk to our team.
What can I alert on?
Cost, latency, error rate and token count. For quality, Online Evaluation alerts on pass rates, and alert rules can sit on top of Agent Insights metrics.
How quickly do alerts fire?
Rules evaluate in real time as traces arrive, with no polling delay — notifications arrive within seconds of the threshold being breached.
Where are alerts delivered?
Email, to one or more recipients, and Slack — through the API with a bot token and channel, or through an incoming webhook URL.
Can I set alerts for one customer?
Yes. Add a Tenant ID filter to a rule and it fires only for that customer — useful for SLA monitoring and budget enforcement.
What's the difference between trace and span scope?
Trace scope watches a whole request end to end. Span scope watches individual operations, such as a single LLM call or tool execution.
How long does setup take?
A few minutes: create a contact point, write a rule with a metric and threshold, and send a test. The alerts quick start walks through it.
Start today
Hear about it before your customers do
Set your first alert rule in minutes and route it to the channel your team already watches. Free to start.