Jev-directed Model Routing

But we have classifiers at home. Why do we need Jev?

Traditional ML classifiers work well when you know the task and can train for its labels. Jev lets you give one model the current state and ask a new, specific question(s) in plain language, then get a typed answer with probabilities that your code can use.

That makes it practical to add many small judgments to a workflow without building a separate model and dataset for each one. And the best part is that you don’t need a whole ML team to do it. The idea of zero-shot classification isn’t new… Jev’s value is making this style of decision a reusable part of software, and that’s what we’re doing with it here.

Spend less on the routing decision

An agent session’s token bill depends on how much context it carries, which models do the work, caching, retries, and how many steps the task takes. These results don’t establish an immediate saving on an individual session. We measured one part of that bill… the model call used to choose where the next request goes.

The dollar amounts below use are using each providers published token prices - they aren’t prices for Agent Router Enterprise.

At 1,000 routing decisions, our benchmark projected roughly 3 cents for Jev versus $1.61 for GPT-6 Sol using uncached token rates, a difference of about $1.58. Across 1,000 developers making that many decisions each month, the annual volume reaches 12 million decisions: about $322 for Jev versus $19,281 for Sol, or 98.3% lower routing token costs and an annual difference of about $18,959. This illustrative workload assumes one routing call per decision; actual savings depend on usage. The comparison below lets you explore personal, business, and enterprise volumes.

We provided each router with the same eight example task states and four predefined execution routes. For Jev, these were supplied as structured state and a typed Choice question, with criteria describing when each route applies. We repeated each task five times, recorded token usage, and applied published prices with no caching discount. The comparison above scales the average cost of valid responses to the selected monthly workload. This comparison covers only the routing decision. The work that follows has its own cost.

How much delay does routing add?

The six-router four-route test used eight authored task states, each repeated five times, for 40 attempts per router. A state included the latest message, recent steps, a prior plan, and verifier limits. Jev returned a typed Choice; the five LLM routers used a 512-token output cap and were asked to return parseable routes. The chart also retains four older Go models measured with a 96-token cap at a different time and with a different request format. These are separate cohorts, not one ten-model capture. No agent executed the requested work.

Before testing, we wrote down which route we expected for each example and gave the models rules for choosing between the four routes - those rules formed our test routing policy. In the six-router cohort, Jev, Astra and Sol matched 40 of 40; Luna matched 39 of 40; Opus matched 36 of 40. Sonnet matched 35 of its 39 valid decisions, with one additional response truncated at the 512-token cap. Its latency percentiles use those 39 valid responses, not all 40 attempts. In the separate older Go cohort, both Grok models matched 40 of 40; DeepSeek and MiniMax matched 35 of 40. This measures agreement with our authored policy, not task success.

To determine if that decision was effective, we would need to run the work and check its results to know whether those choices led to successful tasks… that part of our experiment is already underway and you’ll be able to read about it soon.

A separate two-label comparison

We then tested eight different requests three times each, choosing routine or reasoning. Extraction and formatting were routine; debugging, concurrency, architecture, and proofs needed reasoning. Each router received the same request and policy.

Jev made the decision 12.3 times faster than Luna and 19.8 times faster than Sol, comparing medians. That’s about 1.6 to 2.6 seconds less waiting before dispatch, not a faster completed task. All three returned 24 usable decisions matching our expected routes.

Here Jev was 11.8 times faster than Sonnet and 11.2 times faster than Opus. The first 128-token Claude run left six empty Sonnet responses and one empty Opus response; the 512-token run completed every decision.

So far, it’s working

When using LLMs to direct routing (in an effort to save time/cost), choosing a model is work you pay for before the actual task begins. In these examples, Jev followed our routing rules with less waiting than the general-purpose models. The earlier cost comparison also shows how small savings on that decision can add up across a team.

That gives us a concrete place to use Jev: make the routing decision cheaply, then spend the larger model budget on the work that needs it. Each LLM routing decision adds cost and delay. Across repeated requests, those small overheads add up, creating an opportunity for Jev to reduce both.

We still need to test whether the selected models deliver good results on real workloads. Those results could change our conclusion. For now, the initial evidence supports using Jev to reduce the cost and delay of choosing where the work goes. We’ll repeat these measurements as Jev develops to see whether those benefits improve.