# Metering tokens when the bill is the product

Billing per message is easy and wrong: one user sends a sentence, another uploads a report. Metering the tokens you actually spend is harder, and the hard parts are idempotency, allowance checks and what to do mid-conversation.

By Elson Tan (https://elsontan.com)
Published: 2026-07-28
Reading time: 4 min
Tags: LLM, Cost, Architecture
Canonical: https://elsontan.com/blog/metering-llm-tokens-for-billing/

> Bill on input plus output tokens rather than message count, because message cost varies by orders of magnitude. Record every provider call as a usage event with a stable idempotency key, check the remaining allowance before expensive work starts, and enforce limits at a boundary the user understands.

---
Two customers, same plan, same price. One asked short questions and cost me almost nothing. The
other uploaded 40-page contracts all day and cost me more than they paid.

I was charging per message. A message from the first customer and a message from the second were
the same line on the invoice and nowhere near the same line on my bill.

Message count is a proxy for cost that stops tracking cost the moment your product does anything
interesting. So I moved to tokens, which is what the providers charge me for.

## Count model calls, not user messages

One question from a user is not one call to a model. It is a routing call to decide who handles it,
three rounds of tool use, and a summarisation at the end. Five calls, five bills from my provider.

So that is what I record: one row per model call, tagged with the tenant, the agent and the role it
was playing. Adding them up for an invoice is easy afterwards. Throwing the detail away first is
not something you can undo.

```ts
type UsageEvent = {
  idempotencyKey: string   // stable per provider call
  tenantId: string
  agentId: string
  role: 'orchestrator' | 'classifier' | 'summariser' | 'reasoner'
  model: string
  tokensIn: number
  tokensOut: number
  at: number
}
```

The role tag paid for itself immediately. Aggregate token counts told me I was spending a lot.
Per-role counts told me a third of it was going to a routing step whose entire output was a
sub-agent name, which is the finding that led to routing that role to a cheaper model.

## Why does the idempotency key matter so much?

Because writes fail after the work is done, and a retry without a key charges twice.

The sequence that bites: the model call succeeds, you write the usage event, the write times out,
your retry logic runs it again. Now the tenant has been billed for one call twice. Nobody notices
until a customer reconciles their invoice, and at that point every number you have produced is
suspect.

The key has to be derived from the call, not generated at write time, or a retry produces a fresh
key and the guard does nothing. I use the provider's response id where one exists and a hash of
the request otherwise. Stripe's [idempotent requests](https://docs.stripe.com/api/idempotent_requests)
document the same contract from the other side.

Do this from the first line of billing code. Adding idempotency to a metering pipeline that has
been running for months means auditing every event you have already written.

## Check the allowance before you spend it

Metering tells you what happened. It does not stop a tenant blowing through their plan, and
recording an overage you cannot collect is not a billing system.

Anything expensive gets an allowance check before it starts. A website import that will cost around
51,000 tokens checks that estimate against the remaining balance and refuses up front, rather than
failing partway with half an import written and a partial charge.

| Check | When | On failure |
| --- | --- | --- |
| Hard balance | Before any model call | Refuse, prompt to top up |
| Estimated cost | Before a bulk operation | Refuse with the estimate shown |
| Soft threshold | On crossing 80% | Notify, do not block |
| Plan feature limit | On configuration change | Refuse, explain the plan limit |

The soft threshold is the one that reduces support load. A tenant who gets a warning at 80% tops up.
A tenant whose agent goes silent mid-conversation opens a ticket.

## Where the limit bites is a product decision

A conversation is not atomic. When the balance runs out three tool calls into a five-call task, the
technically correct answer is to stop immediately, and it produces a terrible experience: a
half-finished task and no explanation.

I check at turn boundaries and let an in-flight turn complete, accepting a small overage. The
overage is bounded by the cost of one turn, which is a number I can look at and decide is
acceptable. It is not free and I would not pretend the alternative is obviously wrong. It is a
choice between a small unbilled cost and a user watching an agent stop mid-sentence.

Storage for counters wants low-latency reads on every request. I use
[Workers KV](https://developers.cloudflare.com/kv/) for the wallet, which is eventually consistent,
so two concurrent requests can both see enough balance. Under-counting by one turn is a trade I
took deliberately over putting a strongly consistent read on the hot path of every message.

## Connecting it to money

Usage-based billing has a shape that predates AI, and Stripe's
[usage-based billing](https://docs.stripe.com/billing/subscriptions/usage-based) documentation
covers the meter and subscription mechanics.

The AI-specific parts are that your cost basis moves under you, and that tokens are not a unit
customers have intuition about. When a provider changes prices or you route a role to a cheaper
model, your margin changes without anything in your billing code changing. Store the model and the
provider on every usage event so you can recompute historical margin. Without that you can see what
you charged and not what it cost you.

## Sources

- [Idempotent requests](https://docs.stripe.com/api/idempotent_requests), Stripe API reference
- [Usage-based billing](https://docs.stripe.com/billing/subscriptions/usage-based), Stripe documentation
- [Workers KV](https://developers.cloudflare.com/kv/), Cloudflare
- [AI Gateway](https://developers.cloudflare.com/ai-gateway/), Cloudflare