# Tetrate Agent Router: Frequently Asked Questions

> Frequently asked questions about Tetrate Agent Router covering MCP, providers, pricing, deployment, compliance, and comparisons to LiteLLM, Portkey, and Kong.

Source: <https://tetrate.io/faq>

Last updated: July 2026

## Getting started & capabilities

### What is Tetrate Agent Router?

Tetrate Agent Router is an enterprise AI gateway that routes traffic from your AI agents to any model provider through a single, OpenAI-compatible API. It handles provider failover, token budgets, cost attribution, and governed tool use. It is provider-neutral, framework-agnostic, and sits in front of your existing stack rather than replacing it. It is built on Envoy AI Gateway, which Tetrate co-created and maintains.

### How hard is it to integrate?

You change one line. The base URL changes from your provider's endpoint to Agent Router — everything else in your code stays the same, because the API is OpenAI-compatible.

### Which model providers does it support?

Agent Router is OpenAI-compatible and provider-neutral, with support across major LLM providers including xAI (Grok), Groq, DeepInfra, and open-weight models, alongside the major commercial providers. You can route across providers and configure automatic failover between them.

### Does Tetrate Agent Router support MCP (Model Context Protocol)?

Yes. Agent Router includes a native MCP gateway that gives agents governed access to tools through curated MCP catalogs, with per-profile authentication and tool filtering set centrally by admins.

### Does it support bring-your-own-key (BYOK)?

Yes. BYOK is live — you can use your own provider credentials and switch between Tetrate-managed keys and your own keys.

### Which tools and frameworks does it integrate with?

Agent Router provides in-app integration guides for popular AI coding assistants and agent frameworks, including Aider, Cline, Roo Code, and Goose. Because it is OpenAI-compatible, it also works with any framework that targets an OpenAI-style endpoint.

### Does Tetrate Agent Router support multi-agent workflows?

Yes. Agent Router sits as an OpenAI-compatible gateway in front of agents and multi-agent frameworks, so each agent or workload can authenticate with its own API key — and with Enterprise SSO identity where configured. It does not replace your agent framework; it governs model and tool access for whatever agents you already run.

### Can I self-host or run it on-premises?

Yes, with Agent Router Enterprise. Enterprise runs as a dedicated instance with a distributed data-plane model: you can deploy data planes in your own AWS, Azure, or GCP VPC, on-premises, at the edge, or per-region — all governed from a single Tetrate-managed control plane.

### Do you support hybrid, on-premises, and air-gapped deployments?

Yes for hybrid and on-premises with Agent Router Enterprise: the data plane runs in your Kubernetes environment (any cloud VPC or on-prem), while the management plane stays Tetrate-hosted. The data plane opens an outbound connection to the management plane for configuration only — AI traffic never leaves your environment, and no inbound firewall openings are required for that control path. A fully disconnected air-gapped topology (no outbound connectivity) is not publicly documented; [contact Tetrate](https://tetrate.io/contact/demo-request) if that is a requirement.

### Why use a third-party gateway outside our stack?

It doesn't sit outside your stack. In the Self-Hosted Data Plane deployment, the gateway runs inside your own infrastructure, so every prompt, response, and credential stays within your perimeter. It sits in front of your model providers as a single OpenAI-compatible endpoint, so existing agent code works unchanged. What you gain is the traffic-engineering layer (routing, failover, budgets, guardrails, per-team cost attribution) that would otherwise require building and operating a platform yourself. Tetrate Agent Router is built on [Envoy AI Gateway](https://tetrate.io/envoy-ai-gateway), co-created and maintained by Tetrate with Bloomberg, with a C++ data plane production-tested at Netflix, Lyft, and Airbnb.

## Products & naming

### What is the difference between Agent Router Service and Agent Router Enterprise?

Agent Router Service is the self-serve, developer-focused tier — sign up and start routing through a Tetrate-managed service in minutes. Agent Router Enterprise includes everything in Service, plus cross-team cost attribution with showback and chargeback, admin controls for model and MCP access profiles by team, runtime AI Guardrails, and enterprise SSO where every request carries authenticated identity — all in a dedicated instance.

### What is the AI Gateway?

The AI Gateway is Agent Router's model-routing component: an [approved model catalog](https://tetrate.io/smart-routing), unified provider access, and automatic failover. (You may see it called the "LLM Gateway" in some older materials — it is the same component, now standardized as the AI Gateway.)

### How does smart routing choose a model?

Smart routing in Agent Router Enterprise picks a model for each request in one of three ways: you choose from an approved catalog, your rules choose by request tag, or Tetrate-auto chooses with a decision model. Policy runs first, so no choice can reach a model the team is not allowed to use. See [smart routing](https://tetrate.io/smart-routing).

### Is Agent Operations Director still a product?

Its capabilities are now part of Agent Router Enterprise. The runtime visibility, governance, and policy-enforcement features formerly delivered as Agent Operations Director are now included in Agent Router Enterprise.

### What is the difference between Tetrate Agent Router and Envoy AI Gateway?

Envoy AI Gateway is the open-source data plane. Tetrate Agent Router is the productized, managed, and governed product built on top of it — by the team that co-created and maintains the project. See the full comparison: [Tetrate Agent Router vs. self-hosting Envoy AI Gateway](https://tetrate.io/learn/ai/tetrate-vs-envoy-ai-gateway).

### What is the difference between Tetrate's gateway products (TEG, TIS, Service Bridge)?

These are Tetrate's application-networking line, distinct from the AI products. Tetrate Enterprise Gateway for Envoy (TEG) is a hardened Kubernetes ingress built on Envoy Gateway. Tetrate Istio Subscription (TIS) provides enterprise support for upstream Istio and Envoy. Tetrate Service Bridge (TSB) is an Istio-based service mesh management plane. Agent Router is the AI gateway, built on Envoy AI Gateway.

### What is Tetrate PRISM?

[Tetrate PRISM](https://tetrate.io/prism) is an AI ROI agent for engineering leaders. Leaders ask questions in plain English, and PRISM answers from the company's own data. It joins AI spend from Tetrate Agent Router Enterprise with engineering output from systems such as GitHub, Jira and ServiceNow, and shows the results by team and by person.

## Comparisons

### Is Tetrate Agent Router a LiteLLM alternative?

Yes. Both expose an OpenAI-compatible endpoint, but Agent Router is built on Envoy for production scale, with cost attribution, guardrails, and audit. See [Tetrate Agent Router vs. LiteLLM](https://tetrate.io/litellm-alternative-agent-router-enterprise).

### Is it a Portkey alternative?

Yes — and notably an independent one, since Portkey was acquired by Palo Alto Networks and is now part of Prisma AIRS. See [Tetrate Agent Router vs. Portkey](https://tetrate.io/learn/ai/tetrate-vs-portkey).

### Is it a Kong AI Gateway alternative?

Yes. Kong extends its API-management platform to AI via plugins; Agent Router is AI-native on Envoy AI Gateway. See [Tetrate Agent Router vs. Kong AI Gateway](https://tetrate.io/learn/ai/tetrate-vs-kong).

### How does it compare to Bifrost or Cloudflare AI Gateway?

See the head-to-heads: [vs. Bifrost](https://tetrate.io/learn/ai/tetrate-vs-bifrost) and [vs. Cloudflare AI Gateway](https://tetrate.io/learn/ai/tetrate-vs-cloudflare), or the full landscape in [Best Enterprise AI Gateways 2026](https://tetrate.io/learn/ai/best-enterprise-ai-gateway).

## Provenance & open source

### Who created Envoy AI Gateway?

Engineers at Tetrate and Bloomberg started Agent Router, formerly Envoy AI Gateway, first announced in October 2024 and released as v0.1 in February 2025. It is the first open source AI gateway built on Envoy, started as an Envoy sub-project in the CNCF and now hosted by the Agentic AI Foundation. See [Who created Agent Router?](https://tetrate.io/envoy-ai-gateway)

### Is Tetrate the maintainer of Envoy AI Gateway?

Tetrate is a founding maintainer and maintains the project today, alongside Bloomberg and the broader Envoy community. Tetrate is also a driving force behind Envoy itself.

### Is Envoy AI Gateway backed by the CNCF?

It started as an Envoy sub-project in the CNCF and is now hosted by the Agentic AI Foundation, part of the Linux Foundation. Same code, same maintainers, same Apache 2.0 license.

## Performance

### What latency does Tetrate Agent Router add?

For MCP tool calls, published benchmarks show overhead that is negligible relative to multi-second LLM reasoning time. See the methodology and figures on the [AI Gateway benchmarks page](https://tetrate.io/learn/ai/ai-gateway-benchmarks).

### How does it perform under load?

Agent Router is built on Envoy Proxy, designed for efficient concurrent request handling, giving it a more deterministic memory profile and improved stability under sustained load than Python-based proxies. See the [benchmarks page](https://tetrate.io/learn/ai/ai-gateway-benchmarks) for details.

### What are the scale limits (routes, requests per second)?

There is no single hard product maximum for routes or requests per second — capacity depends on your deployment footprint, and Enterprise sizing is scoped in the subscription. Published Envoy AI Gateway benchmarks cover control-plane route scale and data-plane latency under load; see the methodology and figures on the [AI Gateway benchmarks page](https://tetrate.io/learn/ai/ai-gateway-benchmarks).

## Pricing & commercial

### How much does Tetrate Agent Router cost?

Agent Router Service is usage-based (pay-as-you-go) — you can sign up with a GitHub or Google account and start immediately. For Agent Router Enterprise pricing, [contact us to request a meeting](https://tetrate.io/contact/demo-request).

### How is Tetrate Agent Router Enterprise priced?

Tetrate Agent Router Enterprise pricing is predictable. It is set by inputs you choose up front: the feature set you enable, the size of your deployment footprint (data planes, regions, and environments), and your support and SLA level. It is not metered on token or request volume, so your cost is known before you deploy and does not swing as usage grows. Provider and model costs are settled separately with your provider, so gateway cost and model spend stay independent. The self-serve Tetrate Agent Router Service tier is pay-as-you-go at model cost plus a small routing fee, while Agent Router Enterprise is a scoped subscription. Small teams can start on an entry-level monthly subscription that packages the core governance features at a single-team footprint on a fixed monthly rate, with a clear path to scale up. Contact [Tetrate for a quote](https://tetrate.io/contact/demo-request) based on your feature set and footprint.

### Is there a free way to try it?

Yes. You can sign up for Agent Router Service and start using it right away, with free credit to get started. [Start building](https://router.tetrate.ai/sign-in).

### How do I get started?

Sign up for Agent Router Service with a GitHub or Google account, point your base URL at Agent Router, and start routing. For Enterprise, [book a demo](https://tetrate.io/contact/demo-request).

## Security & compliance

### What security certifications does Tetrate have?

Tetrate maintains SOC 2 Type II and ISO 27001 certifications and is GDPR compliant. Full details, including the Trust Center, are on the [Trust & Security page](https://tetrate.io/trust).

### How is multi-tenant isolation handled?

Agent Router Service is a multi-tenant managed SaaS offering. Agent Router Enterprise runs as a dedicated instance; within Enterprise, projects isolate teams and apps for model access, API keys, and budgets. With a Self-Hosted Data Plane, prompts and responses stay in your environment. See also the [Trust & Security page](https://tetrate.io/trust).

### How is data encrypted?

All data is encrypted in transit using TLS 1.3 and at rest using AES-256, with end-to-end encryption for sensitive communications. See the [Trust & Security page](https://tetrate.io/trust) for the full security practices.

### How does Tetrate Agent Router handle sensitive data and PII?

Agent Router Enterprise includes runtime AI Guardrails that perform PII detection and redaction on every request and response, along with prompt filtering and policy enforcement, before requests leave your network. Tetrate also follows data-minimization practices, collecting only the data necessary to provide the service.

### Does it support data residency?

Yes. With Agent Router Enterprise's distributed data-plane model, you control where each data plane runs — your own cloud VPC, on-premises, at the edge, or per-region — so AI traffic stays where your requirements demand. Tetrate respects data residency requirements and regulations.

### Does every request carry authenticated identity?

With Agent Router Enterprise, yes — enterprise SSO with MFA means every request carries authenticated user and team identity, enforced through role-based access controls and least-privilege principles. This is what enables per-team attribution and audit.

### Do you support SCIM user provisioning?

Not today. Enterprise SSO (SAML and OIDC) provisions users on first login, with optional IdP claim-based role mapping. [Contact Tetrate](https://tetrate.io/contact/demo-request) if SCIM is a requirement for your environment.

### Can complex regulatory policies (GDPR, HIPAA) be expressed and enforced?

No product makes an organization compliant on its own. Agent Router provides technical controls that support regulatory programs: data-plane placement for residency, inline PII/PHI detection and redaction, authenticated identity on requests, audit logs, and curated model and MCP catalogs. [Region rules](https://tetrate.io/smart-routing) in smart routing add in-region failover for data residency. Tetrate maintains org-level GDPR compliance alongside SOC 2 Type II and ISO 27001; for healthcare, the gateway can enforce HIPAA-oriented controls while BAAs with model providers remain your responsibility. See the [Trust & Security page](https://tetrate.io/trust) and [HIPAA AI gateway for healthcare](https://tetrate.io/learn/ai/hipaa-ai-gateway-healthcare).

### Do guardrails interoperate with external providers (for example Azure AI Content Safety or Amazon Bedrock Guardrails)?

Yes. Agent Router Enterprise can call out to Azure AI Content Safety for advanced moderation, and can enable provider-native safety controls such as Amazon Bedrock Guardrails where the upstream provider supports them. Enterprise also supports custom local rules (PII, keyword, regex, and ML patterns). [Contact Tetrate](https://tetrate.io/contact/demo-request) for the current partner and integration list.

### How does Tetrate handle security monitoring and incidents?

Tetrate runs 24/7 security monitoring with SIEM-based threat detection, regular penetration testing and vulnerability scanning, and established incident-response procedures with defined SLAs. See the [Trust & Security page](https://tetrate.io/trust) for details.

Still have questions? [Contact us](https://tetrate.io/contact) or [start building](https://router.tetrate.ai/sign-in).
