Attribute-based LLM routing in Tetrate Agent Router
Attribute-based routing is the rules-based way of smart routing in Tetrate Agent Router Enterprise. Tag a request with one header and let ordered rules pick the model. It sends batch work to cheap models, tiers by plan, and canaries a new model on one cohort, inside your project's policy.
Roy Prins 
Smart routing in Tetrate Agent Router Enterprise picks the model for each request. You choose it, your rules choose it, or Tetrate-auto chooses it, inside the policy you set. Attribute-based routing is the “your rules choose” way. The application tags each request with one header, and the platform team writes ordered rules that pick the model from those tags. The routing change happens at the gateway, with no redeploy.
Sooner or later, someone asks why a request ran on a particular model. A summarization job at 2 a.m. does not need the model that answers your best customer. An agent that plans once and then takes thirty small steps does not need a frontier model for every step. A new model needs real traffic before anyone trusts it, but not all of the traffic.
Most teams answer these with code. Someone adds an if statement, someone else hardcodes a second model name, and the routing policy ends up spread across a dozen services where nobody can audit it. The platform team owns the budget and the approved model list, and has no say in any of it.
Attribute-based routing moves the decision to the gateway. The application says what kind of request it is sending. The platform team says which model that kind of request gets. Either side can change its part without waiting on the other. This post covers how to roll it out without moving a single request until the numbers look right.
Attribute-based routing is generally available in Agent Router Enterprise and is on by default from v0.11.0. On v0.7.0 through v0.10.x an operator must enable it first. The full setup is in the smart routing guide.
How attribute-based routing works
The feature has three parts: an attribute, a header, and a policy.
An attribute is a named definition with a source and a set of allowed values. You declare it once in the Admin Console, and declaring it changes no traffic. The only source is metadata, which means the calling application sets the value. Rules refer to attributes as metadata.workload or metadata.plan.
In the Admin Console, attributes live under Catalog → Request Attributes. A Named set attribute accepts only the values you declare. A Free text attribute lets any caller-supplied value match.
The application sends attributes in a single request header, X-Tars-Metadata, as comma-separated key=value pairs:
POST /v1/chat/completions
Authorization: Bearer <api-key>
X-Tars-Metadata: workload=batch
{ "model": "opus", "messages": [ ... ] }A policy is an ordered list of rules plus a default. Each rule has a condition and a destination model. The router checks rules from top to bottom, and the first match wins. If nothing matches, the request gets the default. Routing never rejects a request.
Conditions use eq, ne, in, and nin, and a rule can hold several conditions that must all hold. A matched rule overrides the model the request named. In the example above the caller asked for opus, and the rule decides what actually serves the request.
A policy attaches to a project or to a single API key. A key’s own policy replaces its project’s policy, default included, so one workload can have its own rules without touching anyone else’s.
Policies live under Policies → Attribute-Based Routing. The form asks for a project, whether the policy covers every request in the project or only requests made with one API key, a required default model, and the ordered rules. A target can have one attribute-based routing policy.
Where it sits in the request path
Routing runs inside your policy. The router loads the policy for the API key, removes the models the project does not allow, and only then applies your rules. A rule can only choose among models the project already has, and its destination has to be valid on the endpoint the request arrived on, or the rule is skipped. A routing rule cannot reach a model the team could not reach by asking for it directly.
That ordering is why it is safe to hand to application teams. They can tag requests however they like, and a wrong tag can at worst send a request to another model the project already approved.
Four patterns to set up first
These come from the scenarios in the guide. Each uses one attribute and a short rule list.
Make overnight work cheaper
Declare an attribute called workload with the values batch and interactive. In a project that grants Sonnet, Haiku, and Flash-Lite, write one rule: when metadata.workload is batch, route to Haiku. Set the default to Sonnet.
Attribute: workload (batch, interactive)
Rule 1: metadata.workload is batch → haiku
Default: → sonnetThe nightly job adds X-Tars-Metadata: workload=batch. Interactive traffic never sends the header, so it takes the default, and nobody has to edit the interactive path.
Serve customer tiers different models
Declare plan with the values trial, retail, and wealth. Order matters here, so put the most valuable tier first:
Attribute: plan (trial, retail, wealth)
Rule 1: metadata.plan is wealth → opus
Rule 2: metadata.plan is retail → sonnet
Default: → haikuTrial customers and untagged requests fall through to Haiku. Specific rules go above general ones, and the default catches the rest. The model you pay for now follows the value of the customer.
Plan big, execute cheap
An agent plans once and then takes many small steps. Tag each call with a task attribute (planning, execution). Planning goes to Opus, execution goes to Haiku, and anything else falls to Sonnet. One agent run uses two models, and the expensive one appears only in the planning step.
Test a model on one cohort
Declare cohort with the values beta and ga. The application tags beta users, and a single rule sends only them to the model under evaluation. Everyone else stays on the current model through the default. That is a canary by cohort instead of by random percentage, which helps when you want feedback from specific users or a like-for-like comparison. Traffic splitting fits better when a random share is what you want.
The same shape retires a model in stages. Send workload=batch to the replacement first. Once it holds up, change the default to the replacement and delete the rule.
Roll it out in monitor mode first
A wrong routing rule in production costs money or quality, so every policy starts in monitor mode. Requests go exactly where they would have gone without the policy, while the router evaluates your rules and records what each one would have matched. Switching to enforce is a separate action on the existing policy, and the audit log records it.
The order that works:
- Define the attribute and save the policy in monitor mode. No traffic moves.
- Ship the header from the application. Still no traffic moves.
- Read the monitor counts for each rule and compare them with what you expected the workload to be.
- Switch to enforce.
Step 3 is the one that pays off. If the nightly job turns out to be a tenth of the traffic you assumed, or one service never sends the header, you find out from a counter instead of an invoice.
A policy saved in enforce mode while the feature was off starts moving traffic when you upgrade to v0.11.0, when the feature turns on by default. Review enforce-mode policies before you upgrade.
Reading the decision record
Each governed request carries response headers that say what happened:
| Header | Value | Present |
|---|---|---|
x-tars-attribute-routing-outcome | matched, default_no_attributes, default_no_match, or default_rules_skipped | Every governed request |
x-tars-attribute-routing-policy | The policy identifier | Every governed request |
x-tars-attribute-routing-rule | The matched rule identifier | matched only |
The request log holds the full record: the policy, its scope and mode, the matched rule and position, the attributes evaluated, the model the caller asked for, the destination chosen, and the model that finally served the request, with latency and cost. In the Admin Console these decisions are labeled Routing rule and Routing default, separate from Budget fallback and Provider fallback. When someone asks why a request cost what it did, the record names the rule.
If a rule seems not to fire, check the outcome header first. A value of default_no_attributes on a request that did send the header means the header did not parse.
What it does not do
Know these limits before you build a plan on it.
- It steers but does not restrict. The caller supplies the attribute values, so a tag is a hint from an application you trust, not an access control. To keep a group off an expensive model, leave the model out of the project’s catalog.
- It ignores provider health and budgets. If a provider degrades or a team hits its limit, use fallback and budget policies. Routing rules stay deterministic.
- It reads only the header. The request body is not inspected. If you want the router to read the request and decide, use Tetrate-auto.
- Matched attribute values are written to the request log, which every organization administrator can read. Keep user identifiers and other personal data out of them.
- Malformed entries are dropped, not refused. The header takes up to 8 entries, keys of 1 to 32 characters, values of 1 to 64 characters, and 2 KiB in total. Anything outside those limits is discarded and counted in
router_attribute_routing_header_dropped_total, and the request takes the default. - Agent Router does not total cost per policy or alert when a rule stops matching. Watch the monitor counts and the request log.
Where to start
Start with the workload that bothers your finance owner most. For many teams that is a batch job: the traffic is predictable, it tolerates a cheaper model, and adding one header to one job is a small change.
Declare the attribute, save the policy in monitor mode, and see what the traffic looks like. Then enforce. Once that works, add the next rule above the default.
Attribute-based routing is generally available in Tetrate Agent Router Enterprise. Start with the smart routing guide and the routing policy reference, or request an enterprise trial.
How attribute-based routing compares with other routing
Smart routing offers three ways to choose a model. Attribute-based routing is the middle one. Traffic splitting and fallback sit beside it for other jobs.
| Method | Who picks the model | Use it when |
|---|---|---|
| Model choice | You choose from an approved catalog | The caller should pick a model from a list the admin set |
| Attribute-based routing | Your rules choose from a request tag | Known workload types, plans, or cohorts need different models |
| Tetrate-auto | The router chooses from the request | The model should match each request without a fixed tag |
| Traffic splitting | A weighted share of requests | You want a random percentage canary or cost mix |
| Fallback | The next healthy or in-budget model | A provider degrades or a team hits a budget limit |
Frequently asked questions
How does attribute-based routing relate to smart routing?
Smart routing in Tetrate Agent Router Enterprise picks the model for each request in one of three ways: you choose it, your rules choose it, or Tetrate-auto chooses it. Attribute-based routing is the “your rules choose” way. The application tags the request, and ordered rules pick the model inside the policy you set.
What is attribute-based routing?
Attribute-based routing selects the model for a request from tags the application sends in the X-Tars-Metadata header. A policy holds an ordered list of rules and a default. The first matching rule wins, and a request that matches no rule gets the default.
How does the application tag a request?
It sends X-Tars-Metadata with comma-separated key=value pairs, for example workload=batch,task=summarization. Attributes must be declared in the Admin Console with their allowed values, or they match nothing.
Can a routing rule send a request to a model the project is not allowed to use?
No. Routing selects only among models the project already has. A rule whose destination is not valid on the endpoint the request arrived on is skipped.
What happens when no rule matches?
The default model serves the request. Every policy requires a default, and routing never rejects a request. The x-tars-attribute-routing-outcome header reports default_no_match or default_no_attributes.
How do I test a policy before it affects traffic?
Every policy is created in monitor mode. Requests are unaffected while match counts are recorded per rule. When the counts look right, switch the policy to enforce. The audit log records the change.
Can I set routing rules for one API key?
Yes. Policies attach to a project or to a single API key. A key’s policy replaces its project’s policy entirely, including the default.
What is the difference between attribute-based routing and traffic splitting?
Traffic splitting sends a weighted share of requests to each model, whatever the request is. Attribute-based routing chooses by what the application says the request is, so you can route a specific workload, plan, or cohort.
Does attribute-based routing react to provider outages or budgets?
No. Use fallback and budget policies for those.
Agent Router is Tetrate's Envoy-based AI data plane — AI Gateway, MCP Gateway, and AI Guardrails — built on Envoy AI Gateway by its creators, and kept continuously patched by the Tetrate Patch Service.
References
- Smart routing guide, Agent Router Enterprise docs
- Routing policy reference, Agent Router Enterprise docs
- Smart routing: three ways to choose a model
- Tetrate Agent Router Enterprise: product overview
- What is LLM routing?: methods and trade-offs
Roy Prins 
