Something uncomfortable has been circulating among engineering teams in recent months: Claude Code sends 33,000 tokens before it even reads your prompt. OpenCode does the same with 7,000. That difference isn't trivial — it's a bill that silently compounds with every inference cycle, across every developer on the team, across every working day.
And that's not the only signal. Teams migrating production agents to newer models are reporting 27% cost reductions and 2.2x speed improvements. Which means the model they committed to nine months ago — the one that looked like state of the art — had been overcharging them without anyone auditing it.
The pattern is always the same: a tool gets chosen at the peak of the hype cycle, embedded in the development workflow, declared "productivity" and filed away. The token counter never stops spinning.
AI developer tools get evaluated on their demos, not on their behavior under sustained production load.
The real cost isn't the per-token price: it's the accumulated systemic cost plus the architectural dependency it creates.
Visibility over consumption and quota isn't a luxury for large teams — it's basic hygiene for anyone using AI tools daily.
Tokens: The Currency Nobody Counts Properly
When a developer evaluates an AI tool, the typical criteria revolve around response quality. Does it complete code well? Does it understand the repository context? Does it hallucinate less than the alternative? Those are legitimate questions. But there's a dimension that almost never enters the initial evaluation: how many tokens does the tool consume just to start up.
Token overhead isn't a minor implementation bug. It reflects deep architectural decisions: how much context the tool loads by default, how it manages conversation memory, what metadata it sends on every call. A tool that sends 33k tokens of preamble before reading your instruction isn't just "expensive" — it's a tool that has chosen breadth of context over operational efficiency. That might make sense for some use cases. For a team of ten developers using the agent eight hours a day, the cumulative cost is structural.
The tool isn't bad. Nobody measured what would happen when you scaled it from a prototype to a real team.
The lack of real-time visibility over consumption and quota makes this worse. Teams discover they've burned through their monthly plan mid-sprint — not because of abusive usage, but because per-call overhead was never in their cost mental model. Token spend monitoring should be as standard as latency monitoring or CPU utilization. It rarely is.
The context window problem: more isn't always better
There's an implicit belief that a larger context window is always preferable. It's intuitively appealing: more context, better understanding, better responses. But the context window has a direct cost per token and an indirect cost in latency. A model processing 200k tokens takes longer and costs more than one processing 20k, even when those extra 180k tokens add nothing for your specific task.
The right question isn't "which tool has the biggest window?" but "how much context does my task actually need?" For most code assistance operations — completing a function, reviewing a block, generating a test — the relevant context is local and bounded. Loading the entire repository on every call isn't being smarter: it's being more expensive.
The Accelerated Obsolescence Cycle
There's another tension emerging with serious implications beyond per-token cost: AI skills for developers expire faster than any other technical skill. And with them, so do integrations.
A team that invested in mastering a particular model's API, in building optimized prompts, in tuning the behavior of their CI/CD agent, discovers six months later that the model has changed its behavior, a new version breaks previous assumptions, or a cheaper and faster competitor has appeared. Integration debt accumulates silently.
This isn't new in technology — every technical dependency has a lifecycle — but the rate of change in the model ecosystem is qualitatively different. In traditional infrastructure, a major version of a database comes every two or three years. In the LLM space, the model that's state of the art today can be the expensive, mediocre option in twelve months. We already see it in migrations reporting 27% performance gains just from updating the underlying model of an agent.
The practical consequence is that taking AI to production with real criteria means designing integrations for change, not for stability. Not vendor lock-in disguised as "native integration." Abstraction layers that let you swap the underlying model without rewriting business logic. Your own benchmarks against your real tasks, not the generic benchmarks the vendor publishes.
The vendor benchmark trap
Model rankings published by vendors — and replicated by the community with standard datasets — measure performance on generic tasks. Coding benchmarks like HumanEval or SWE-bench are useful references, but they don't tell you how the model behaves in your codebase, with your conventions, solving your types of task.
A team primarily working with a proprietary framework, unindexed internal documentation, or domain-specific code patterns needs its own evals. This isn't weeks of work: with two or three days of effort you can build a representative test set that lets you compare models on real cases. That's the only benchmark that matters for adoption or migration decisions.
Technical Sovereignty: The Question Nobody Asks
Behind the cost and obsolescence debate there's a more structural question that very few teams ask when adopting an AI tool: what happens if this vendor changes their terms, raises prices, or simply decides your use case isn't a priority?
Technical sovereignty in AI isn't just a debate for nation-states and regulators. It's a legitimate question for any company starting to build critical workflows on top of third-party APIs. If your automated QA pipeline, your documentation assistant, or your bug triage agent depends on an external API with terms of service that can change unilaterally, you've introduced systemic risk that rarely appears in the initial adoption analysis.
This connects directly to something we routinely confuse with resilience: a system that works well today isn't necessarily a robust system. Dependence on a single AI vendor can look like operational stability while the service works. The problem surfaces when the vendor decides to change their default models, deprecate an API, or raise prices by 40%.
The options aren't binary — use third-party AI or use nothing — but they do require a conscious decision about which workflows can depend on external providers and which need alternatives. Open-source models running on owned infrastructure have matured enough to be a real option in many mid-market enterprise scenarios, especially for internal tasks where absolute performance matters less than predictability and data sovereignty.
How to Audit Your AI Tools Before the Bill Surprises You
You don't have to wait until the problem shows up on the invoice. There's a set of questions any team can ask today about the AI tools they're already using — or evaluating for adoption.
The first is about visibility: do you know exactly how many tokens each typical operation in your workflow consumes? Not the number in the vendor documentation: the real number in your logs. If you don't have that instrumentation, you don't have the information to decide.
The second is about dependency: how much effort would it take to swap the underlying model in your integration? If the answer is "a lot, because we've built on assumptions about the current model," you have integration technical debt that deserves to be in the backlog.
The third is about evaluation: are your adoption decisions based on generic benchmarks or on tests against your real use cases? Vendor benchmarks are well-intentioned technical marketing. Your own evals are the only signal that matters.
And the fourth — the most ignored — is about planned obsolescence: when did you last compare the model you use in production against today's alternatives? If the answer is "when we adopted it," you're probably paying a comfort tax that nobody has quantified.
The tool you chose nine months ago isn't competing against the tools of nine months ago. It's competing against today's.
At Room 714 we work with teams that want to make these decisions with real technical and economic criteria, not by inertia or hype pressure. If you have AI tools embedded in your development workflow and have never done a proper cost and dependency audit, now is a good time to do it. The token counter doesn't wait.
