What your AI feature really costs, and why half of AI companies don't know

30 June 2026 · 6 min read

For twenty years, software COGS was the boring line. Hosting, support, payment processing. It moved slowly, it scaled sub-linearly, and nobody built a career on watching it. Gross margins of 75–80% were the default, and finance teams priced products as if serving one more customer cost approximately nothing.

Then you shipped an AI feature, and the boring line started moving.

The problem: COGS became a variable cost nobody assigned an owner

Every AI request has a marginal cost. Not a metaphorical one — an invoiced one, denominated in tokens, that scales with usage rather than with revenue. A customer who pays you £40 a month can consume £4 of inference or £400, and your pricing page does not know the difference.

The industry data says this is not a rounding error. In ICONIQ Capital's 2026 State of AI: Bi-Annual Snapshot, a survey of roughly 300 executives at companies building AI products, respondents expect AI product gross margins to reach roughly 52% on average in 2026. That is an improvement on prior years — and it is still 20 to 30 points below the SaaS baseline those same companies are valued against.

Now the uncomfortable half of the story. Mavvrik's State of AI Cost Governance Report, a 2025 survey of 372 companies, found that half of companies with AI-core products do not track LLM API costs at all. The same survey found 84% of companies report AI costs eroding gross margins. Read those together: the cost is large enough to move margins at 84% of companies, and half of the companies most exposed to it are not measuring it.

That is the state of the industry. A margin-defining cost line, and a coin-flip chance the company incurring it knows what it is.

Why your dashboards mislead you

"But I have dashboards" is the standard objection. You do. They mislead you in three specific ways.

1. Per-provider silos. Your usage lives in the OpenAI console in dollars, the Anthropic console in dollars, an OpenRouter page for the models you route there, and possibly a cloud bill for embeddings. No single number exists. Adding them up requires a spreadsheet, a currency conversion, and an afternoon — so it happens quarterly, if at all.

2. No revenue join. A provider dashboard can tell you that you spent $6,200 last month. It cannot tell you which customers, plans or features that spend served, because the provider has never heard of your customers. Spend without a revenue join is a weather report. Cost per account joined to revenue per account is a margin.

3. Totals hide the distribution. Provider dashboards report calendar-month aggregates. Unit economics live in the tails, and the tails are invisible in an aggregate.

A worked mini-example

A support-automation product charges £40.00 per seat per month, 1,400 seats, with an LLM behind every conversation. Total AI spend last month: £5,862.22 across three providers. Blended, that is £4.19 per seat — an 89.5% gross margin on the feature. The dashboard looks fine. Everyone moves on.

Here is the same month with the revenue join and the distribution:

| Cohort | Seats | AI cost per seat | Revenue per seat | Gross margin | | --- | --- | --- | --- | --- | | Bottom 90% of usage | 1,260 | £2.10 | £40.00 | 95% | | 90th–99th percentile | 126 | £18.40 | £40.00 | 54% | | Top 1% | 14 | £64.13 | £40.00 | −60% |

Fourteen seats cost more to serve than they pay. On its own, £897.82 of negative-margin usage is survivable. The problem is where those seats sit: heavy users cluster in your largest accounts, which are precisely the accounts your sales team wants to expand. Every expansion deal on this pricing copies the loss at a bigger size. The blended 89.5% number told you none of this. The distribution is the finding; the average is the anaesthetic.

This is also why "we'll optimise later" fails quietly. Later arrives as a renewal negotiation with your biggest customer, in which usage-based pricing is suddenly your idea and their objection.

What to do about it

None of this requires a data team. It requires four habits.

  1. Inventory every key. One list: every API key, which provider, which product feature, which environment. If a key serves two features, split it. Attribution starts at the key, because the key is the only identifier the provider's usage data will ever give you.
  2. Pull usage daily, into one currency. Provider usage APIs exist and are free to read. Normalise to the currency your accounts are kept in, at a recorded rate. Monthly is too slow — a retry loop can burn a month's budget in a weekend.
  3. Join cost to revenue. Cost per account, next to revenue per account, every week. This is the only view in which "is this feature profitable" is a question with an answer.
  4. Watch the tails, not the mean. Set a threshold — say, any account whose cost exceeds 30% of its revenue — and review the list weekly. It will be short. It will also be where all the news is.

The pattern in the data is consistent: the companies with 52% margins and the companies that cannot see their margins are, per Mavvrik, often the same companies. Measurement does not fix a margin by itself. But nobody has ever fixed one without it.

TokenLedger does the mechanical part — key inventory, daily sync from every provider, one ledger in your currency, joined to Stripe revenue per account. The judgement part stays yours.