Tetrate Is Bringing Agent Router, Powered by Envoy, to the Agentic AI Foundation

Why Agent Router

The market is rapidly filling with LLM and MCP gateways, from coding-agent companies and data platforms to AI startups and cloud providers entering the space. The need is real: as enterprises deploy more agents, use frontier and open-weight models, and connect varied SaaS and local MCP servers and tools, they need a control point to manage and govern all this AI traffic.

However, enterprises don’t want N different gateways: one tailored for one coding agent, one embedded in their data platform, another for SaaS models, and another for local MCP tools.

They need one organizational AI traffic layer that can work across the entire ecosystem, regardless of the agent framework, coding agent, model provider, MCP tool, dataset, or guardrail engine.

That layer must be more than an experimental, harness-specific proxy that just shipped last quarter. It needs to be reliable, scalable, observable, secure, and battle-tested infrastructure that enterprises can trust.

Agent Router, built on a battle-tested Envoy proxy, brings the proven traffic layer to AI and provides that foundation.

What is different about AI traffic

AI traffic differs from traditional application traffic in a few key ways.

• Traffic is highly spiky. Agent workloads can generate unpredictable bursts of requests.
• AI backends are inherently variable. Model providers can experience capacity constraints, outages, rate limits, or regional availability issues, making intelligent routing and failover essential. Recent availability incidents with Anthropic models validate this.
• Sessions are long-lived, and context matters. Routing decisions may need to preserve session affinity and understand context across multiple interactions.
• Requests are unstructured. AI payloads often contain prompts, tool calls, context, and other dynamic content that requires deeper inspection than traditional HTTP traffic.
• Identity and policy follow the agent and user. The same user or application may invoke different models, tools, and data sources within a single workflow, making consistent identity, authentication, authorization, and policy enforcement critical.
• The ecosystem is constantly changing. New models, agents, MCP servers, tools, and AI frameworks are emerging every day. The traffic layer must remain independent of any particular vendor or application, yet work well with all of them.

Why now

Agents are becoming critical infrastructure as they move into core business workflows. Envoy has been battle-tested for a decade across thousands of organizations and is already trusted as critical infrastructure for managing massive volumes of application and service traffic.

Agent Router brings that proven traffic-management foundation to the rapidly evolving world of AI agents.

Enterprises need this proven traffic layer so that agent calls to models and MCP tool calls pass through it, and so that the layer can’t change shape while your agent is in production.

Agent Router’s API is stable and mature. Version 1.0 shipped in June 2026 with a compatibility commitment for the 1.x series, and version 1.1 followed in August. Agent Router arrives at AAIF as mature software, not a new project. It has eleven publicly listed adopters, including Bloomberg, Tencent Cloud, Nutanix, LY Corporation, and the National Research Platform, along with 132 contributors from 21 organizations and fifteen stable releases.

As Tetrate brings Agent Router to AAIF, the community can expect it to serve agent builders better and offer a seamless path into the broader agentic ecosystem. While core Envoy will continue to add native support for AI protocols like MCP and A2A, which are already in AAIF, Agent Router will enable their use by agent builders and operators through integrations with agent SDKs, coding agents, model providers, MCP and tool providers, and guardrail engines. Moving the project from an Envoy sub-project to a standalone AAIF project puts it in the same agentic ecosystem as the supporting projects, like MCP, A2A, and Goose, and lets Agent Router become central to the rest of the AI ecosystem faster.

How to use Agent Router

Day 0: get started

Agent Router runs on Envoy, but you don’t need an existing Envoy deployment to use it. Pick the deployment that fits:

Locally. One binary, no Docker, no Kubernetes. The aigw binary reads the environment variables your OpenAI SDK already uses:

# OpenAI
OPENAI_API_KEY=sk-... aigw run

# Ollama
OPENAI_BASE_URL=http://localhost:11434/v1 OPENAI_API_KEY=unused aigw run

# Front your MCP servers
aigw run --mcp-config mcp-servers.json

As a service. Access the hosted version of Agent Router, with a UI for observability and troubleshooting, at router.tetrate.ai to get started with already enabled integrations.

With Goose and coding agents. With Goose (an existing AAIF project), you can log in to Tetrate Agent Router Service, select your model, and start using Goose.

From your agents or coding harnesses. Agent Router speaks the OpenAI API. Point the SDK at it and change nothing else:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:1975/v1", api_key="unused")

To switch models, change the model name in the request. Agent Router routes it to whichever provider serves that model. Most coding agents read the same two environment variables. Set them, and the agent’s traffic gets failover, budgets, and tool authorization without touching the agent itself:

export OPENAI_BASE_URL=http://localhost:1975/v1
export OPENAI_API_KEY=unused

In Kubernetes, as a single instance. Run it as a dedicated gateway for AI traffic, installed with the Envoy Gateway quickstart. You use the same configuration as the local binary, expressed as Kubernetes resources: AIGatewayRoute for routing, AIServiceBackend for each provider, BackendSecurityPolicy for credentials, and MCPRoute for tools.

As a centralized governance platform with a multi-dataplane and multi-cloud setup. Tetrate runs Agent Router Enterprise, which lets every team’s agents go through one governed entry point, with credentials, budgets, and tool access set per tenant. This is how enterprises run it. See the Tetrate Agent Router Enterprise docs.

What is ahead

Agent Router will be how you talk to the Envoy data plane for the behavior you want, and it will bring the developer experience, ecosystem integrations, and usability. The Envoy community continues to solve data-plane challenges in Envoy itself, while Agent Router provides the control plane that manages Envoy for AI traffic. Agent Router complements Envoy.

Get involved

We welcome contributors who care about developer experience, including quickstarts, error messages, the standalone binary, and docs, not only proxy internals. The most valuable contribution is a production problem the community hasn’t seen yet.

• Discord. Questions, design discussion, and release news.
• Community meeting. Every Monday, open to everyone. Bring your use case.
• In person. “Your Agents Need a Router: One Integration for Every Model and Tool” at AGNTCon + MCPCon Europe, Amsterdam, Thursday 17 September, 13:10 CEST, and at AGNTCon + MCPCon North America, San Jose, Friday 23 October, 12:45 PM PDT.
• Contributing. Read the roadmap, open a design proposal, or pick up a good first issue on GitHub. Contributions use DCO sign-off.

Read the AAIF announcement and the technical introduction to the Agent Router project.