AI Gateway for Kubernetes and EKS: How the Gateway Fits Your Cluster and Your Coding Tools
Last updated: October 2026
Tetrate Agent Router Enterprise fits the cluster and the coding tools you already run, before each team wires up its own provider keys.
An AI gateway for Kubernetes installs into your existing cluster as a standard Helm chart or GitOps manifest. Tetrate Agent Router Enterprise uses your cloud IAM, sends metrics and traces to the monitoring tools you already run, and needs no inbound connection from Tetrate. Coding tools such as Claude Code, Cursor, and GitHub Copilot connect by changing a base URL and an API key. A command line tool writes that config for the common ones.
Platform teams usually ask two questions in the same meeting: will the gateway fit our cluster, and will our developers switch to the gateway?
An AI Gateway for Kubernetes Installs as a Helm Chart in Two Namespaces
The self-hosted data plane is a set of ordinary Kubernetes resources. The architecture overview describes an Envoy-based gateway plus a Controller that polls the Tetrate-hosted management plane over one outbound HTTPS connection.
| Part | Value | Purpose |
|---|---|---|
Namespace tars-system | Controller | Polls configuration and applies policy |
Namespace tars-dataplane | Gateway | Carries AI traffic |
Service egress, port 10080 | Gateway entry point | Serves /v1/* and /mcp/* |
GatewayClass tars-egress | Envoy Gateway v1.7.x | Standard Kubernetes Gateway API |
| Kubernetes version | 1.32 to 1.35 for current releases | See the compatibility page |
Images ship for linux/amd64 and linux/arm64. The installation overview lists three install paths:
| Path | When to use the path |
|---|---|
The tare CLI | Managed clusters on EKS, AKS, or GKE. The CLI bundles the chart and runs checks before install. |
| Plain Helm | Private registries. The chart comes as an OCI artifact. |
| GitOps | Argo CD or Flux. tare install renders the manifests for you to commit. |
On EKS, the Install Uses Your Load Balancer and Your IAM Roles
The AWS installation guide asks for at least three nodes, OIDC turned on, and the EBS CSI driver. The smallest setup is three m5.large nodes, and the largest documented setup is three m5.2xlarge nodes. You copy images to ECR, and one tare install command deploys the data plane.
The tare gateway install command makes the gateway reachable through a Network Load Balancer. The default path needs no AWS Load Balancer Controller and no IAM policy work. You add a CNAME to the load balancer’s hostname, and you’re done.
For Bedrock, the data plane signs requests with the pod’s IAM identity through IRSA or EKS Pod Identity. No static AWS keys sit in the cluster. One command binds the role: tare dataplane identity bind --type irsa --create-iam --cluster <name> --region <region>.
Two setup tips. Pin the Kubernetes version to the supported range when you create the EKS cluster. And give the gateway a dedicated cluster for its GatewayClass.
Metrics and Traces Go to the Monitoring Tools You Already Run
The gateway exports traces over OTLP on gRPC port 4317 or HTTP port 4318 to Grafana, Datadog, New Relic, Honeycomb, or Jaeger. Each component also exposes a Prometheus scrape endpoint. Spans carry the model name, token counts, and the request ID, using OpenInference attribute names. Set contentMode: metadata-only in the Helm values and prompt text stays out of your traces.
Your gateway configuration can live in Git too. The configuration-as-code guide explains that policies, routing, budgets, the model catalog, and API keys are the same objects in the console, the API, and Helm values. Changes apply in under a minute without a restart.
Two setup tips. Set each value in one place, either Git or the console, so your GitOps tool and your admins agree. And place your telemetry collector Service in a namespace the gateway watches, as the telemetry guide shows.
Claude Code, Cursor, and Other Coding Tools Connect by Base URL and API Key
Every tool that accepts an OpenAI-compatible or Anthropic-compatible base URL can use the gateway. The coding agents guide covers the current list.
| Tool | How the tool connects |
|---|---|
| Claude Code | tare integrate claude-code writes the base URL and key to ~/.claude/settings.json. Passthrough mode keeps a Claude Max subscription as the billing path and still logs every request. |
| Claude Desktop, Cowork and Code tabs | tare integrate claude-cowork, in gateway mode. See Roy Prins’ Claude Desktop post. |
| Visual Studio Code, including GitHub Copilot chat and agent mode | The Tetrate VS Code extension registers the gateway as a model provider. One key serves every extension that asks VS Code for a model. |
| Cursor | A base URL setting, plus a .cursorrules file for generated code |
| Codex, Aider, Cline, Continue, Coder | tare integrate <tool> writes the tool’s config file |
We built tare integrate so nobody copies base URLs and keys by hand. The command finds the tools installed on the machine and creates or reuses a gateway key. Then the command writes each config file with 0600 permissions, backs up the old file, and makes one test call. Add --dry-run to see the change first.
Claude Code Reaches Every Model Family Through One Gateway Profile
Claude Code sends requests in the Anthropic Messages format. The gateway accepts that format on /v1/messages and translates each request for the provider behind the model. Since release 0.3.0, one Claude Code profile reaches Anthropic, Claude on Vertex AI, OpenAI, Gemini, and self-hosted models, with no flag changes per model. Your own tools and MCP servers work on every family, because the gateway converts the tool definitions for each backend.
| What you’ll see | Why it happens | What to do |
|---|---|---|
| OpenAI rejects a large MCP setup | Claude Code sends more tool definitions than OpenAI accepts | Set ENABLE_TOOL_SEARCH=true, so Claude Code searches its tools instead of sending all of them |
An enabled model is missing from /model | Claude Code filters the picker by its own naming rules | Start Claude Code with --model <name> |
| Anthropic server-side tools fail on other models | Those tools run inside Anthropic only | Keep that work on Anthropic models |
| You want to change model family | Each session works best with one model family | Start a new session for the new family |
Passthrough mode covers Anthropic models only, because the subscription pays only for Anthropic. Passthrough fallback stays inside Anthropic, and the console shows that limit.
For aliases, canary splits, and how to confirm which model answered, see how to switch LLM models safely.
Microsoft Copilot Is Several Products, and Most of Them Route Through the Gateway
Four Microsoft products can send their model calls through the gateway today: GitHub Copilot chat and agent mode in VS Code, the Copilot CLI, Azure AI Foundry agents, and Copilot Studio agent tools. To bring your own models into Microsoft 365, use a custom engine agent or a Foundry agent published to Microsoft 365. The full product-by-product table is in Which AI Tools Can an AI Gateway Govern?
For Azure AI Foundry, three settings decide whether the connection works, per Microsoft’s bring-your-own-model guide:
| Setting | Value |
|---|---|
| Connection type | Other source in the portal, ModelGateway in the CLI and Bicep |
| Base URL | The base path only. Foundry adds chat/completions itself. |
| Model name in the agent | <connection-name>/<model-name> |
Two setup tips. Connect Foundry prompt agents, which use the Chat Completions API that this connection expects. And turn on bring-your-own-model in your Copilot Business or Enterprise policy before you set a rollout date.
Applications Connect the Same Way as Tools
An app built on the OpenAI SDK, the Anthropic SDK, LangChain, Vercel AI SDK, Pydantic AI, or CrewAI changes the base URL and key. The integrations reference lists the full set. Any tool with a base URL setting connects the same way.
How to Get Started
Everything on this page ships in Agent Router Enterprise today. Start with the installation overview and the coding agents guide. To see the install on your own cloud, request a demo.
Agent Router Enterprise
Tetrate Agent Router Enterprise routes AI agent traffic across providers and your own models, with policy, cost controls, and audit on every request — in cloud, on-prem, or edge.
Frequently Asked Questions
Can I install an AI gateway into my existing EKS cluster? Yes, if the cluster has at least three nodes, OIDC turned on, and a supported Kubernetes version. The install is a Helm chart or a set of rendered manifests. A cluster that already runs a cluster-wide Envoy Gateway needs a separate cluster.
Does a self-hosted AI gateway need inbound access from the vendor? No. The data plane makes one outbound HTTPS connection to the management plane. No inbound firewall rules are needed.
How do I connect Claude Code to an AI gateway? Run tare integrate claude-code. The command writes the base URL and API key to the Claude Code settings file and makes a test call. A passthrough flag keeps a Claude Max subscription as the billing path.
Can Claude Code use OpenAI or Gemini models through an AI gateway? Yes. Claude Code sends Anthropic Messages requests, and the gateway translates each request for OpenAI, Gemini, Claude on Vertex AI, and self-hosted models. One profile covers every family. Anthropic server-side tools stay on Anthropic models.
Can GitHub Copilot use an AI gateway? Yes. Copilot chat and agent mode in VS Code connect through the Tetrate extension, and the Copilot CLI connects through three environment variables.
Does the gateway work with Datadog or Grafana? Yes. Traces export over OTLP, and each component exposes a Prometheus scrape endpoint. A metadata-only setting keeps prompt text out of traces.
Can I manage AI gateway configuration in Git? Yes. Policies, routing, budgets, the model catalog, and keys can be set through Helm values and applied by Argo CD or Flux. Changes apply in under a minute.
How does the gateway call Amazon Bedrock without static AWS keys? The data plane signs Bedrock requests with the pod’s IAM role through IRSA or EKS Pod Identity. One tare dataplane identity bind command creates and binds the role.
Related: Which AI Tools Can an AI Gateway Govern?, on the full Microsoft Copilot and coding tool list; Enabling agentic developers, on developer self-service in VS Code; Adding Tetrate Agent Router to Visual Studio Code, on the VS Code extension.
Learn more about Tetrate Agent Router Enterprise — enterprise AI agent routing with policy, cost controls, and audit across every gateway.
