What Are AI Guardrails? Definition, Categories, and How They Work
TL;DR: AI guardrails are control layers, technical as well as procedural, that sit between human intent and machine interpretation. Unlike prompt engineering, which is a development practice, guardrails are runtime controls that enforce policy on every request. These guardrails are grouped into six categories: input filtering, output filtering, identity, tool access, spend, and residency. At the gateway level, these controls run inline in the data path, intercepting traffic to validate, redact, and log requests before they reach a model or tool. Each verdict is recorded with the caller’s authenticated identity, and changes to the guardrails go to an audit log that no user can edit, so actions can be audited.
Guardrails can run inside each application through a library, in model-native filters, or centrally in a gateway, and each enforcement point covers a different slice of the risk. An in-app SDK sees application context only where developers have adopted it, while a gateway sits in the data path and can see calls routed through it. This guide defines AI guardrails, covers the categories that matter for enterprise security, and maps each control to the request lifecycle.
What AI guardrails are and why auditability matters
An AI guardrails definition separates development practice from runtime enforcement. Guardrails are a layered defense. Each layer enforces a specific policy at a specific point in the request lifecycle.
What separates a runtime control from a policy document
AI guardrails are control layers, technical as well as procedural, that sit between human intent and machine interpretation. The procedural layer covers policy, review, and approval workflows. The technical layer enforces those policies at runtime, inspecting and controlling traffic as it flows.
The distinction is auditability. A policy document records the intended behavior. A technical guardrail produces a log showing what was blocked, redacted, or allowed, giving an auditor verifiable evidence.
Preventing unauthorized data exposure
The specific risk is PII, secrets, and regulated data leaving the network. Inline redaction and policy enforcement in the data path can catch data before it reaches a model.
A model may not follow a system prompt. Runtime guardrails enforce policy in the data path independent of the prompt. Prompt injection targets the gap between what the prompt asks and what the system allows.
Classifying AI guardrails by risk control
The taxonomy maps guardrail categories to the risks they mitigate.
| Category | Risk Target | Example Controls | Enforcement Point |
|---|---|---|---|
| Input filtering | PII, injection attacks, malformed prompts | Regex masking, token replacement, schema validation | Before inference |
| Output filtering | Data leakage, toxic content, hallucinations | Content filtering, consistency checks, PII redaction in response | At model output |
| Identity | Unauthorized access, shadow AI | Authenticated identity on every request, role-based access | Before entry |
| Tool access | Excessive agency, unauthorized actions | Curated tool catalog, per-profile permissions, operator-approved tool lists | At tool call |
| Spend | Unbounded consumption, runaway costs | Token limits, rate limiting, budget enforcement | Inline in request path |
| Residency | Data sovereignty violations | Data plane deployment in customer network, localized processing | Infrastructure deployment |
PII redaction and request sanitization
PII redaction strips or masks sensitive data before inference. Request sanitization goes further, validating the structure and content of the request itself. The mechanism is regex masking, token replacement, and policy rules applied inline.
Input filtering guardrails screen user inputs before they reach the model, catching PII, credentials, and injection attempts at the boundary before the data enters the model’s context window.
Output filtering and content validation
You need both input and output filtering because risks exist in both directions. Input filtering catches PII before it reaches the model. Output filtering catches PII or disallowed content in the model’s response.
Models can generate sensitive data even when the prompt is clean. Output validation guardrails evaluate model responses before they reach the end user. Both layers are necessary because a model can repeat memorized training data, such as contact details, in a response to a prompt that contains none.
Verifying user and agent identity
Identity should be carried on every request. The mechanism is an authenticated identity attached to each call, so the gateway knows who made the request and can apply per-user policy.
Identity on every request answers the auditor’s question of who accessed what. Role-based guardrails tie the AI system to the existing identity and access management layer, enforcing which knowledge bases, tools, or actions a given role can access, so sensitive context stays within authorized boundaries.
Restricting AI tool call permissions
Agents call tools. A guardrail that governs only model calls leaves tool calls ungoverned. The mechanism is a curated tool catalog with per-profile authentication and filtering.
Effective guardrails for AI agents use allowlists by agent identity, tool argument schemas, and tool value limits. Requiring human approval for destructive tool usage like refunds or deletions adds a procedural layer to the technical control.
Usage quotas and rate limiting
Spend and rate limits are guardrails. The mechanism is token limits enforced inline in the request path, so the control applies before the cost is incurred.
Token-aware cost caps and circuit breakers can stop unbounded consumption before it becomes a billing incident.
Enforcing data residency for AI models
Data residency is a guardrail category. The control is running the data plane in the customer’s own cloud, on-premises, or at the edge, so prompts and data stay inside the network. In hybrid deployment, the customer runs the data plane in their own environment while Tetrate hosts the management plane.
Request log data control determines what reaches the management plane. The customer configures log capture to keep full request bodies inside the perimeter while exporting metadata for billing and monitoring.
How runtime guardrails intercept AI requests
Runtime guardrails intercept, validate, redact, or rewrite traffic inline in the data path between the application and the model.
The management plane is not in the data path. Each data plane keeps a local copy of the rules, so AI traffic keeps running when the management plane is unreachable. During an outage, rule changes pause, but enforcement continues. This architecture is designed so that governance does not become a single point of failure.
Mapping the AI request lifecycle
Every AI request moves through five stages. Guardrails apply at specific points in this flow.
- Request entry: Authentication, authorization, and initial validation.
- Prompt processing: PII redaction, injection detection, and sanitization.
- Inference: The model generates a response.
- Output processing: Content filtering, PII redaction in the response, and consistency checks.
- Tool call: Permission verification against the approved tool catalog before execution.
Defining guardrail enforcement zones
Each zone in the lifecycle answers a specific risk question. Mapping guardrails to zones helps provide layered coverage.
| Zone | Risk Question | Guardrail Category | What Gets Logged |
|---|---|---|---|
| Before entry | Is this user or agent authorized? | Identity | Authentication result, user ID, timestamp |
| Before inference | Is this prompt safe and clean? | Input filtering | Redaction actions, blocked patterns |
| At output | Is this response safe to return? | Output filtering | Filtered content, consistency flags |
| At tool call | Can this agent perform this action? | Tool access | Tool called, arguments, approval status |
Enforcement points for secure AI operations
The gateway is the central enforcement point where all zones can be enforced inline on live traffic. A gateway sits in the data path and sees traffic routed through it, including traffic from shadow AI tools that bypass application-level controls. It also carries identity, tenant, and data-class context that model-level controls do not see.
Validating requests before entry
At entry, the gateway checks authentication, authorization, rate limits, and the model catalog. The mechanism is identity carried on every request, with policy evaluated before the request proceeds.
Every agent request passes through the same pipeline inside the AI gateway. Each stage runs in order and either returns a response or passes the request on. This creates a single control point between your agents and model providers.
Sanitizing prompts before inference
Before inference, the gateway redacts PII, detects prompt injection, and enforces prompt controls. The mechanism is inline regex masking and token replacement before the prompt reaches the model.
Redaction rules live in the gateway, not in application code. A platform operator sets them once for a project, and a changed rule applies everywhere in that project as soon as it takes effect.
Securing data at the model output stage
At output, the gateway filters content, redacts PII in responses, and runs consistency checks. The mechanism is output scanning in the data path, with a monitor mode that logs without blocking during rollout.
Runtime guardrails enforce boundaries at execution time: the model generates a response, and the gateway authorizes, constrains, or blocks the action before it reaches the user or triggers a tool call. This execution-time enforcement matters when the LLM can call tools and trigger operations.
Validating agent tool usage in real time
At tool call, the gateway checks the tool catalog, verifies the agent’s permissions, and logs the call. The mechanism is per-profile authentication and filtering, ensuring agents only access authorized tools.
An MCP Gateway sits between AI agents and MCP servers. Agents connect to the gateway, which acts as a centralized control layer for tool traffic. This gives security teams a single point to audit what an agent reached.
Linking user identity to guardrail policy
Guardrail policy is only as strong as the identity it is tied to. Identity on every request means the gateway can apply per-user policy and log who did what.
Callers authenticate to the gateway with an API key issued for their project or, in preview, a token from the organization’s identity provider, so every tool call has a user, team or customer attached. This identity binding lets access decisions reflect who is making the request.
Verifying agent access with audit records
Two records show who did what. Policy decision records capture each allow, block, modify, or redact verdict with a correlation ID, the policy version in force, and the caller’s identity. The audit log records administrative changes, such as new API keys or guardrail edits, and no user can edit or delete its entries.
Tetrate has completed a SOC 2 Type II audit, is certified to ISO 27001, and is compliant with GDPR. See Tetrate’s compliance certifications for details, and the Trust Center for security documentation.
Granular access control by identity
Access control is per-profile. A user’s profile determines which models and tools they can call. The mechanism is role-based access and scoped access profiles.
Scoped tool access uses allowlists to decide which tools each identity can see and call, so agents only reach the tools their role allows.
Securing agent actions with runtime controls
Agent actions are a high-risk surface. Runtime controls can govern the actions an agent takes.
A gateway that sits between agents and MCP servers can enforce tool-calling governance inline at the moment of the tool call. Tetrate Agent Router Enterprise provides an MCP Gateway with per-profile authentication, an operator-approved tool catalog, and the end user’s identity recorded on every tool call.
Real-time threat blocking and alerts
The gateway blocks disallowed actions in real time and alerts on policy violations. The mechanism is inline policy enforcement with alerting integrated into your existing security operations.
The independent Broadcom/VMware test of the Agent Router project (then named Envoy AI Gateway) measured about 2 ms of gateway overhead per request, about 0.01% of end-to-end latency, under real GPU inference.
PII redaction and audit logging
Redaction and logging work together. The log shows what was redacted and what was blocked.
Configurable data capture lets the customer choose whether to send the full request, only counts and cost, or nothing at all. The setting can be changed at any time. Configurable log capture balances audit depth against data minimization requirements.
Verifying agent permissions in real time
Agent permissions can be checked at the moment of the tool call. The mechanism is step-up authorization for higher-risk actions, with routine actions proceeding without friction.
Step-up authorization, built with an identity orchestration provider (Ory), lets routine agent actions run freely while high-risk actions trigger human approval through the customer’s own identity system.
AI guardrails are a layered defense, and the gateway is the enforcement point where all layers can be applied inline with an audit trail. Tetrate Agent Router Enterprise is built on the Agent Router project (formerly Envoy AI Gateway), an open source data plane at the Agentic AI Foundation (AAIF) that runs on the Envoy proxy. Agent Router Enterprise runs gateways in any cloud, region, on-premises site, or edge location, and manages all of the gateways from one management plane. Requirements can cover input filtering, output filtering, identity, tool access, spend controls, and residency enforcement, all verified through policy decision records tied to the caller’s identity and a tamper-evident audit log of administrative changes.
Start a fast-track evaluation: first routed request in about 15 minutes after onboarding, nothing to install. Request a demo to see runtime guardrails enforced inline, with a policy decision record for each request, on live traffic.
Agent Router Enterprise
FAQs
What’s the difference between input and output guardrails?
Input guardrails screen prompts before they reach the model, catching PII and injection attempts. Output guardrails validate model responses before they reach the user, catching data leakage and unsafe content.
Can guardrails stop agents from accessing unauthorized tools?
Yes, when implemented at the gateway with a curated tool catalog. The gateway checks each tool call against per-profile permissions and blocks unauthorized access attempts in real time.
Do guardrails slow down AI requests?
In an independent Broadcom/VMware test of the Agent Router project, overhead measured 2 ms per request, roughly 0.01% of end-to-end latency under sustained load.
How do you audit what guardrails actually blocked?
Through policy decision records. Each blocked, modified, or redacted request produces a record with a correlation ID, the policy version in force, and the caller’s identity, so you can see what was blocked, when, and under which policy. A separate audit log records changes to the guardrails themselves, and no user can edit or delete its entries.
Key terms glossary
AI guardrails: Control layers, technical as well as procedural, that sit between human intent and machine interpretation, enforcing policy on every request at runtime.
Input filtering: Guardrails that screen user prompts before they reach the model, catching PII, injection attacks, and malformed requests.
Output filtering: Guardrails that evaluate model responses before they reach the end user, catching data leakage, toxic content, and inconsistencies.
Identity: Authenticated user or agent identity carried on every request, enabling per-user policy enforcement and audit trails.
Tool access: Controls governing which external tools and APIs an AI agent can call, enforced through a curated catalog with per-profile permissions.
Spend: Token limits and rate limiting enforced inline in the request path, preventing unbounded consumption by misbehaving or looping agents.
Data residency: The requirement that prompts and data remain within a specific network perimeter or geographic region, enforced through deployment architecture.
MCP Gateway: A centralized control layer for Model Context Protocol traffic that provides authentication, authorization, and observability for agent tool calls.
Step-up authorization: A mechanism that allows routine agent actions to proceed automatically while requiring human approval for higher-risk operations.
Prompt engineering: The practice of designing the instructions given to a model to shape its behavior, applied at development time.
Learn more about Tetrate Agent Router Enterprise — enterprise AI agent routing with smart routing, policy, cost controls, and audit across every gateway.