Reasoning effort is a cost dial, not a quality setting

Every message that arrives at my platform hits a classifier first, which picks the specialist agent to handle it. Small model, short prompt, four possible answers.

It was taking 4.6 seconds.

I checked the model. Fast. I checked the prompt. Short. Nothing about the job explained it, and since this call sits in front of every single message, 4.6 seconds was the floor for the whole product.

Then I read the response payload properly. The model was thinking. Hundreds of tokens of internal deliberation, on a multiple-choice question, because extended reasoning was on by default and I had never turned it off.

You are paying for the thinking

The thinking happens before the answer you see, and you are charged for it. Anthropic bills thinking tokens as output tokens. So does OpenAI with reasoning tokens. Output tokens are the expensive ones.

Which means leaving the default alone is a spending decision, taken once and then applied to every call you ever make, whether or not the work deserves it.

On my classifier it was about ten times the tokens needed to say one word.

Every provider spells it differently

This is the part that makes it easy to ignore. There is no shared parameter, so wiring it up means touching each provider adapter separately.

Provider	Parameter	Values
Anthropic	thinking.budget_tokens	An integer budget
OpenAI	reasoning_effort	low, medium, high
BytePlus	extra_body.reasoning_effort	minimal, low, medium, hi
DeepSeek	Not configurable	Always on for reasoner models
OpenRouter	Passed through	Depends on the underlying model

An integer budget and a three-value enum do not map onto each other cleanly, and pretending they do is how you end up with medium meaning something different on every path. I normalise to four internal levels and let each adapter translate, accepting that the translation is approximate.

type Effort = 'minimal' | 'low' | 'medium' | 'high'

const ANTHROPIC_BUDGET: Record<Effort, number | null> = {
  minimal: null,     // omit the thinking block entirely
  low: 5_000,
  medium: 10_000,
  high: 20_000,
}

export function applyEffort(provider: Provider, body: RequestBody, effort: Effort) {
  switch (provider) {
    case 'anthropic': {
      const budget = ANTHROPIC_BUDGET[effort]
      if (budget) body.thinking = { type: 'enabled', budget_tokens: budget }
      return body
    }
    case 'openai':
      body.reasoning_effort = effort === 'minimal' ? 'low' : effort
      return body
    case 'byteplus':
      body.extra_body = { ...body.extra_body, reasoning_effort: effort === 'high' ? 'hi' : effort }
      return body
    default:
      return body // provider ignores it; not an error
  }
}

The default branch returning the body untouched is deliberate. A provider that does not support the control should be a no-op, not a thrown error, or adding a provider breaks every call site.

Which calls actually need it?

The ones where the model has to work something out. Not the ones where it has to pick from a list.

I set effort per agent role rather than per request, because roles are stable and requests are not:

• Routing and classification: minimal. The output space is a handful of tokens. Deliberation adds latency and changes the answer roughly never.
• Summarising and formatting: low. Structure matters, invention does not.
• Planning and multi-step tool use: medium. Getting the order wrong is expensive downstream.
• Debugging and analysis: high. This is what the budget is for.

Tie the level to the role in the same place you choose the model, and it stays consistent as prompts get edited by people who are not thinking about token budgets.

Measure it, because intuition is wrong here

I would not have found the classification problem by reading code. Aggregate token counts looked normal. Per-role counts, once I tagged usage by role, showed a step that produced four words consuming a large share of the bill.

Route everything through a gateway that records tokens per call with the role attached. Cloudflare AI Gateway does this in front of multiple providers, which is convenient if you are already routing across several.

The number to watch after you turn the dial down is not cost. It is your quality signal for that role: retry rate, escalation rate, validation failures. Dropping a role to minimal and watching escalations climb tells you the reasoning was doing something. Dropping it and seeing nothing move tells you it was not.

My classification path lost nothing measurable at minimal effort. The debugging path would be worse at anything below high, and I have not tried to save money there.

Sources

• Extended thinking, Anthropic documentation
• Reasoning models, OpenAI platform documentation
• AI Gateway, Cloudflare
• Workers AI, Cloudflare