AI Gateway Security: SSO, Private Networks, Encryption, and What Data Is Stored

Last updated: October 2026

Tetrate Agent Router Enterprise answers the four questions every security team asks, before your security review turns into a six-week questionnaire.

AI gateway security comes down to four questions. Who can sign in? What has to be open in your network? Where is traffic encrypted? What data does the platform keep? In Tetrate Agent Router Enterprise, people sign in through your identity provider over OIDC, with no local passwords. A self-hosted data plane makes one outbound HTTPS connection to Tetrate, and nothing connects in. Traffic is encrypted on every hop the gateway controls. With a self-hosted data plane, prompt and response text stays out of Tetrate’s systems unless request logs keep the text. Your admin can turn off that logging with one setting.

Every security team asks these four questions in the first call.

Agent Router Enterprise has no local password database. Single sign-on through OpenID Connect (OIDC) is the only way into the Admin Console and the Developer Console, per the SSO guide. Multi-factor sign-in stays with your identity provider, where you already manage the policy.

Part	How the part works
Supported identity providers	Microsoft Entra ID, Okta, Google Workspace, Ping Identity, Auth0, Keycloak, and any OIDC provider
Accounts	Each account is created automatically at the person’s first sign-in, and the person appears in the Users list
Roles	Group or role claims from your provider map to Agent Router roles. The mapping runs at every sign-in and overwrites the stored role, per the role mapping guide.
First admins	The Admin Emails field names who becomes an admin at first sign-in

Two setup tips. Fill in Admin Emails before the first sign-in, so your first admins arrive with full access. And when someone leaves, update your directory and revoke the person’s API keys together, so access ends on every path at once.

To decide what each role can change, see AI model access control.

The Data Plane Calls Out to Tetrate, and Nothing Calls In

With a self-hosted data plane, the gateway runs in your Kubernetes cluster and Tetrate hosts the management plane. The management plane never opens a connection into your network. The data plane polls for configuration over one outbound HTTPS connection, per the planes and components page. Your gateway address can stay internal.

The prerequisites page lists every outbound destination, all on port 443:

Destination	Why the data plane calls the destination
`api.<tenant>.tetrate.ai`	Configuration and health
`auth.<tenant>.tetrate.ai`, `router.<tenant>.tetrate.ai`, `dashboard.<tenant>.tetrate.ai`	Sign-in and the consoles
`registry.tetrate.ai`	Images, at install and upgrade only. You can mirror the images to your own registry.
Your AI providers and MCP servers	The requests your apps send
Your telemetry backend, if you set one	Metrics and traces

If the connection to Tetrate stops, the gateway keeps routing requests with its last configuration. New settings wait until the connection returns. The data plane also works through your company proxy, with management traffic and model traffic on separate proxy settings, per the corporate proxy guide. Traffic to model providers can stay off the public internet. Use AWS PrivateLink, Azure Private Link, or GCP Private Service Connect, per the network and security page.

Two setup tips. Give Microsoft services a route to your gateway when you use them: Copilot Studio calls an MCP URL from Microsoft’s cloud, and Azure AI Foundry agents reach the gateway inside the Azure network they use. And if your proxy inspects encrypted traffic, plan the proxy settings with Tetrate engineers during your first install.

For the full list of Microsoft Copilot products that route through the gateway, see which AI tools an AI gateway can govern.

Encryption Covers Every Hop the Gateway Controls

Hop	Encryption	Who controls the setting
Your apps to the gateway	TLS at your ingress. Mutual TLS can be required of callers.	You
Gateway to self-hosted models	TLS, with mutual TLS supported	You
Gateway to public AI providers	TLS	The AI provider
Data plane to management plane	HTTPS, outbound only	Tetrate

Tetrate states TLS 1.3 in transit and AES-256 at rest for Agent Router Enterprise, in David Wang’s LiteLLM comparison. For a formal answer in a security review, ask for the compliance reports described below.

What Data the Platform Keeps, and What You Can Turn Off

The management plane stores configuration and usage numbers. Live prompt and response text stays in the data plane, and request logs are where you choose whether to keep that text.

Data	Where the data is kept	Can you change or turn off the data?
Each person’s work email and group claims	Management plane, from your SSO	Required for sign-in
Token counts, cost, latency, status per request	Management plane	Kept in every mode, because billing and usage reports need the numbers
Prompt and response text in request logs	Management plane	Yes. Full is the default. Metadata only drops the text. Off stores no request log.
Audit log of admin actions	Management plane	Kept permanently, and no user can change or delete an entry
Traces and metrics	Your own telemetry backend	Yes. A metadata-only setting keeps prompt text out of traces.

The request log setting is under Settings → Request logs and takes effect in seconds, per the request log guide. Retention and purge are configurable, per the log retention guide. The audit log reference lists each field, including user email, action, resource, time, source IP, and user agent.

Two setup tips. Request logs start in Full mode, so switch to Metadata only before real users send real prompts. And send the request log signal only to telemetry systems approved for prompt data, because that signal carries the full record.

Set Four Controls Before Your First Users Arrive

Control	Where	Why the control matters
Admin Emails	SSO settings	The first admins can sign in
Request log mode	Settings → Request logs	Prompt text stays out of logs from day one
Telemetry content mode	Helm values for the data plane	Prompt text stays out of your traces
Data residency	Routing policy	Requests reach only providers in your allowed region, per the data residency guide

To prove what the gateway decided, export audit and policy decisions to your SIEM, such as Splunk or Datadog.

Pick the Deployment Model That Matches Your Data Rules

Deployment model	Where prompts are processed	Best for
Agent Router Enterprise Fully Managed	Tetrate infrastructure, in a dedicated instance	Teams that want governance without running a cluster
Agent Router Enterprise Self-Hosted Data Plane	Your Kubernetes cluster	Regulated data, private networks, data residency

Tetrate hosts the management plane in both models. The deployment models page compares them. Residency options are US and EU, set per routing policy, so one deployment can carry different rules for different projects, per the compliance reference. SOC 2 Type II and ISO/IEC 27001 reports are available under NDA.

How to Get Started

OIDC sign-in, the outbound-only data plane, proxy support, request log modes, audit logs, SIEM export, and data residency all ship in Agent Router Enterprise today. Start with the network requirements and the SSO guide. For a security review with Tetrate engineers, request a demo.

Agent Router Enterprise

Tetrate Agent Router Enterprise routes AI agent traffic across providers and your own models, with policy, cost controls, and audit on every request — in cloud, on-prem, or edge.

Learn more

Frequently asked questions

Can I use my own SSO with an AI gateway? Yes. Agent Router Enterprise signs people in through your identity provider over OIDC, including Entra ID, Okta, Google Workspace, and Ping Identity. There are no local passwords, and each account is created at the person’s first sign-in.

Does a self-hosted AI gateway need a public endpoint? No. The data plane makes one outbound HTTPS connection to the management plane, and the management plane never connects in. The gateway address can stay internal.

What happens if the gateway loses its connection to the management plane? The gateway keeps routing requests with its last configuration. New settings wait until the connection returns.

Is AI gateway traffic encrypted? Yes. Traffic is encrypted with TLS on every hop. That covers your apps to the gateway, the gateway to AI providers and self-hosted models, and the data plane to the management plane. Mutual TLS is supported toward callers and self-hosted models.

Does an AI gateway store my prompts? Request logs store prompt and response text by default. An admin can switch request logs to metadata only, which keeps token counts and cost and drops the text, or turn request logs off.

What personal data does an AI gateway keep? Agent Router Enterprise keeps each person’s work email and group claims from SSO, and an audit log of admin actions with user, time, and source IP. Prompt text can contain personal data, so set request logs to metadata only if that matters to you.