Smart LLM routing for AI agents

Send Every Request to the Right Model

Tetrate Agent Router Enterprise picks the model for each request. You choose it, your rules choose it, or Tetrate-auto chooses it, inside the policy you set.

Book a Demo

Deterministic Probabilistic

You choose

Model choice

Admins set each team's catalog. Users pick from it.
Your rules choose

Attribute-based routing

Admins set the traffic classes. App owners tag requests.
The router chooses

Tetrate-auto

A decision model picks for each request, inside policy.

What's smart routing

Three ways to choose a model, from deterministic to probabilistic

Start simple and add automation when you are ready.

You choose

Model choice

User → approved catalog → chosen model

Give each team an approved list. Admins assign models to each team. Users pick only from that list.
Keep a group on cheaper models. Create a project with only low-cost models and issue the group's keys there.
Stop hardcoding model names. Publish one name for apps to call. Repoint it when a model changes. No redeploy.

Your rules choose

Attribute-based routing

Request + tag → ordered rules → model

Make overnight work cheaper. Tag batch jobs and route them to a cheap model. The nightly job adds one header.
Serve tiers different models. Tag each request with the plan. Order the rules by value.
Plan big, execute cheap. An agent plans once, then takes many small steps. Tag each call by task, and one run uses two models.
Test a model on one cohort. The app tags beta users. The rule sends only them to the new model.
Retire a model gradually. Move batch traffic to the new model first. When it works, change the default.

The router chooses

Tetrate-auto

Request + context → decision model → best fit

Let the router pick for each request. Apps call tetrate-auto. The decision model reads the request and picks a model.
Match the model to the task. Simple requests go to small models. Hard work, like code review, goes to a frontier model.
Keep every pick inside policy. Admins set limits, such as the most expensive model a dev key may use. Tetrate-auto stays inside them.

How it works

How a request is routed

Five steps, the same for every request. Policy runs first.

Request arrives

The app sends a model name, tags, or both.
Policy applies

The router loads the one policy for this API key and removes the models it doesn't allow.
A model is chosen

Choice: the asked-for model. Rules: the first match. Auto: the decision model's pick.
A model serves it

No match? The default model serves the request.
The decision is saved

Mode, reason, model, and policy.

Inside your policy

A team can't reach a model its project doesn't have, whichever way the model is chosen.
Inside your region

Region rules limit a model to providers in the allowed region, and failover stays there. If no model in the region can serve the request, the request fails in the region.
On the record

Every request records which way chose the model, which model served, and which policy applied. That record answers audit questions and explains every bill.

Business outcomes

What smart routing changes

Four results, each with a number to track.

Cost tiering

Match model price to the value of each request. Routine agent steps stop paying frontier prices.

Track: cost per finished task and the share of calls on a frontier model.

Tenancy tiering

Route by customer, plan, or environment. Your top plan gets the best model. Trials don't.

Track: cost per tenant and per plan.

Model lifecycle

Update, replace, and retire models behind one stable name. No app release when a provider changes a version or a price.

Track: app releases per model change (target: zero).

Region rules

All three ways see only the models allowed in the request's region, and failover never leaves the region. Sovereign AI rules apply to the router too.

Track: cross-region fallbacks (target: zero). Data residency routing.

Example

Smart switching finished more tasks at half the cost per finished task

A decision model chose at each step whether an agent handed work to a cheaper model.

The switching pattern

A A A B B B B B B B B B B A

A Strong model starts and does a short final check.
B Cheap model does the routine work.

Routing rule	Calls	Tasks finished	Cost per finished task	Time
Strong model only	A A A	50%	100%	2.7 min
Smart switch, decision model, timed	A B A	60%	50%	2.0 min
Cheap after step 1, strong check, fixed	A B A	40%	13%	1.9 min
Cheap after step 1, no check, fixed	A B B	30%	17%	2.0 min

Cost is relative to the strong model alone. The two fixed handoffs cost less per finished task but finish fewer tasks, and every unfinished task needs a rerun or a person. Read the smart routing data sheet.

FAQ

Frequently asked questions about smart routing

What is smart routing in Agent Router Enterprise?

Smart routing is the model routing feature in Tetrate Agent Router Enterprise. It sends every request to the right model in one of three ways. You pick the model from an approved catalog, your rules pick it from tags on the request, or Tetrate-auto picks it with a decision model. All three choose only among the models your policy allows, and every decision is recorded.

Can smart routing pick a model a team isn't allowed to use?

No. Policy removes models the team may not use before any choice is made. If a pick falls outside the allowed set, the default model serves the request.

Can smart routing send data out of my region?

No. Region rules limit a model to providers in the allowed region, and failover stays in that region. If no model in the region can serve the request, the request fails and stays in the region.

How does smart routing reduce AI agent cost?

Routine agent steps go to cheaper models and hard steps go to a frontier model. Smart switching finished more tasks than the strong model alone at half the cost per finished task. Measure the result as cost per finished task, because cost per token doesn't count retries and failed steps.

Do I need Tetrate-auto to use smart routing?

No. Model choice and attribute-based routing work without a decision model. Turn on Tetrate-auto when you want the router to pick for each request.