AI ROI Maxxing: How to Report AI ROI to Your Leadership Team

AI ROI has two halves: developer outcomes and inference spend

I build AI gateways, so you can expect me to argue that you need one. I will. But a gateway solves half of this problem at most. Getting your organization to agree on what a meaningful outcome looks like is a leadership job, and no infrastructure will do it for you.

Break the problem into the numerator and the denominator of ROI, and work on each one separately. Return = the outcomes your developers drive. Investment = your total inference spend.

The common mistake I see is treating both halves as one problem and reaching for one solution to cover them. That approach fails every time. One half is organizational and the other is technological, and they need different people and different timelines.

So you need proxies on both sides of the equation. Different kinds of proxies, and a different mindset for each. One you negotiate, the other you deploy.

Most teams spend months on the left column and never touch the right one. Start on the right. Collecting the data costs less than reaching agreement, and the argument about value cannot start without real numbers on a screen.

How to choose AI ROI metrics when no metric is perfect

Leadership will not tell you what they want to see. They will tell you what is wrong with what you brought. Defining the metric was never their job. Representing your work with the right number is yours, so you have to bring one.

The failure mode is waiting for a metric you can defend against every objection. That metric does not exist, so the report never gets written, and the CFO keeps asking.

Engineers will supply the objections. Pull requests measure activity, not value, and a developer who wants a higher count can open smaller ones. Lines of code punish the person who deletes three hundred lines and improves the system. Story points are a planning estimate that teams calibrate differently, so they compare badly across teams. Time saved per developer rests on a guess about how long the work would have taken without AI, which nobody wrote down. Every one of these objections is correct.

You still have to pick one. Make the call, then say where the metric is weak and how far you will let it travel. A metric you understand the weaknesses of is usable. Waiting for a metric with no weaknesses is how a year goes by with nothing reported.

If your organization rejects every proxy for the value of an activity, ask why the activity exists. If a merged pull request tells us nothing about value, what is a pull request for? Nobody wants to answer that, and the question moves the room from “your metric is bad” to “here is a better one.”

The right set of proxy metrics is different for every organization, so there is no list to copy. What matters is that a set exists and that people agree on it, and agreement of that kind only comes from the top.

Label your first report a draft. Say out loud that the metrics are proxies and that you expect to replace them. A review becomes an editing session.

Proxy metrics for developer productivity you can collect this week

Developers ship features and answer support tickets, so their output lands on revenue, customer experience, and operating cost. Pick the metric closest to each.

None of these numbers is the value itself. Each one stands in for value you cannot count directly. Say so in the report, then improve on it.

Worked example: plotting developer output against AI spend

Take the first row of that table and put it to work. Pull requests merged is the proxy we chose for feature delivery, and the objection to it is real: a developer who wants a higher count can open smaller pull requests.

Here is the chart we build. Count of pull requests on one axis, median size of change on the other. One dot per developer, sized by AI spend. No developer controls both axes at once, so the picture is hard to fake.

Then read it by quadrant. Each corner describes a different way of working, and dot size tells you what that way of working costs. A developer sitting in a productive quadrant with a small dot has found something the rest of the team has not, and a large dot in a quiet quadrant is a bill without an outcome behind it.

Treat the chart as a map of working styles. Ranking people by it will mislead you, because each quadrant means one thing at low spend and something else at high spend.

A small dot in a productive quadrant is someone who found an approach the team should copy. A large dot next to few merged changes is a bill with nothing behind it.

×

Why AI spend is hard to track across providers, clouds, and subscriptions

The chart above assumes you know what each developer costs. Most enterprises do not, and the reason is that AI reaches people, and increasingly agents, through many paths at once.

Count the paths in your own organization. Frontier APIs from OpenAI and Anthropic. The same models again through Azure AI Foundry, Amazon Bedrock, and Google Vertex AI, priced differently in each. Open-weight models on your own GPUs. Coding assistants sold by the seat. Claude and ChatGPT subscriptions expensed by individuals. Agent platforms inside SaaS products your teams already pay for. A home-grown gateway one team built last year. And direct API calls from an application that nobody registered anywhere.

Each path reports something, and no two report the same thing.

Now try to assemble a report from those pieces. The billing periods differ. The units differ, since some paths give you tokens, some give you requests, some give you credits, and some give you GPU hours. Identity is the hardest part: a hyperscaler project maps to a workload instead of a person, and a shared key maps to nobody at all. By the time you reconcile a quarter of this by hand, the quarter is over and the numbers have moved.

You were hired to run an engineering team. Collecting and reconciling all of this by hand takes time you need elsewhere, and building your own proxy in front of every provider puts an operational load on your team that grows with each new model and region.

How an AI gateway attributes AI spend to teams, agents, and projects

Tetrate Agent Router Enterprise is an AI gateway built on Envoy AI Gateway. It runs in your environment, between your AI clients and your model providers. Requests from Claude Code, Codex, Cursor, custom harnesses, and internal agents all arrive at one endpoint that speaks the OpenAI and Anthropic APIs. From there the gateway routes to OpenAI, Anthropic, Google Gemini, Amazon Bedrock, or a self-hosted open-weight model.

Developers keep the tools they already use. They open a console, mint their own key, and see the model catalog you approved along with the MCP servers you allow. They also see their own spend and their own caching behavior, which creates useful self-policing and keeps provisioning work off your desk.

×

Administrators get usage analytics with cost, tokens, and request counts over any time range. The Enterprise tier adds breakdowns by team and user, which is what turns raw telemetry into showback and chargeback. Budgets cap spend against the same dimensions before it surprises you.

×

These capabilities are what your ROI report depends on.

Distributed AI gateways with one central management plane

A single gateway in one region stops working once your developers sit in three countries and your customers sit in ten. Requests cross an ocean and come back, and a coding assistant that pauses at every keystroke gets abandoned. Data protection law in some of those countries also decides which region a prompt is allowed to reach.

So you put gateways close to the workloads they serve. Install those gateways as separate systems and you are back where this post started, with each region reporting its own numbers in its own format on its own schedule.

Tetrate Agent Router Enterprise splits the two jobs apart. Gateways run wherever traffic needs handling, on your infrastructure or hosted. Model access, budgets, guardrails, and identity are set once in a management plane that covers all of them, and every region reports into the same analytics. The regional deployment guide covers how traffic moves between them.

The regional split also changes what your report can show. The same task costs different amounts in different regions, because providers, available models, and what local teams prefer to work with all vary. A single global average hides that difference. Cost per outcome broken out by region tells you where the work is cheapest to run.

AI ROI is developers moving across the chart

Go back to the four quadrants. Report AI ROI as movement. Developers cross that chart over a quarter, and the movement is the return.

A developer moves from spending with nothing merged to shipping steadily. Another keeps shipping the same work at a third of the cost after switching models. A third stops one-shotting whole features and starts sending changes a reviewer can read. Each move is small. Together they are the return your CFO asked about, and they are visible on one picture that fits in a board deck.

None of that is measurable while cost sits in eight different systems. Tetrate Agent Router Enterprise closes that gap: every request through one governed path, gateways near the workloads they serve, and cost attributed to a person, a team, or an agent so you can price the move as well as observe it. Set budgets on the same identities and the chart becomes something you steer with.

Then stop building the report by hand. The management plane already has what every gateway collected, across regions and providers, in one schema with identity on each request. Point an agent at it. Ours answers questions in plain language: which teams moved quadrants this quarter, what the move cost, where spend is growing faster than output. Your CFO gets a sentence, and the number traces back to a single request when an engineer asks.

The argument about AI ROI becomes an argument about which direction to move next.

A thirty-day plan to report AI ROI

Then repeat. The version you show in month six will be the one leadership quotes back to you.

Get hands-on with Tetrate Agent Router Enterprise to understand your AI ROI today.

×

Common questions about measuring AI ROI

What is the best metric for AI ROI in engineering? There is no single best metric. Pair one output metric, such as pull requests merged weighted by size of change, with inference spend for the same person or team. The ratio between them is the number to report.

How do you track AI spend across OpenAI, Anthropic, and the hyperscalers? Route every request through one AI gateway that sits between your clients and your providers. The gateway records tokens, cost, and identity in one format, so you stop reconciling separate bills that use different units and billing periods.

Why do AI pilots show no measurable return? Usually because spend and output are measured on different populations. Individual subscriptions, seat licenses, and direct API calls sit outside the reporting path, so the cost side is incomplete and the ratio means very little.

Read the Tetrate Agent Router Enterprise documentation, or start with the fast-track evaluation to see analytics and budgets running on your own traffic.