# Reasoning effort is a cost dial, not a quality setting

Every major provider now exposes a knob for how hard the model thinks. Left at its default it burns tokens on arithmetic. Set per role, it is one of the largest cost reductions available without changing models.

By Elson Tan (https://elsontan.com)
Published: 2026-03-24
Reading time: 4 min
Tags: LLM, Cost
Canonical: https://elsontan.com/blog/reasoning-effort-as-a-cost-dial/

> Reasoning modes are billed as output tokens, so a model left on its default thinking budget can cost several times more than needed on trivial calls. Each provider exposes the control differently, so normalise it behind one interface and set the level per agent role rather than globally.

---
Every message that arrives at my platform hits a classifier first, which picks the specialist agent
to handle it. Small model, short prompt, four possible answers.

It was taking 4.6 seconds.

I checked the model. Fast. I checked the prompt. Short. Nothing about the job explained it, and
since this call sits in front of every single message, 4.6 seconds was the floor for the whole
product.

Then I read the response payload properly. The model was thinking. Hundreds of tokens of internal
deliberation, on a multiple-choice question, because extended reasoning was on by default and I had
never turned it off.

## You are paying for the thinking

The thinking happens before the answer you see, and you are charged for it. Anthropic
[bills thinking tokens as output tokens](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking).
So does OpenAI with [reasoning tokens](https://platform.openai.com/docs/guides/reasoning). Output
tokens are the expensive ones.

Which means leaving the default alone is a spending decision, taken once and then applied to every
call you ever make, whether or not the work deserves it.

On my classifier it was about ten times the tokens needed to say one word.

## Every provider spells it differently

This is the part that makes it easy to ignore. There is no shared parameter, so wiring it up means
touching each provider adapter separately.

| Provider | Parameter | Values |
| --- | --- | --- |
| Anthropic | `thinking.budget_tokens` | An integer budget |
| OpenAI | `reasoning_effort` | `low`, `medium`, `high` |
| BytePlus | `extra_body.reasoning_effort` | `minimal`, `low`, `medium`, `hi` |
| DeepSeek | Not configurable | Always on for reasoner models |
| OpenRouter | Passed through | Depends on the underlying model |

An integer budget and a three-value enum do not map onto each other cleanly, and pretending they do
is how you end up with `medium` meaning something different on every path. I normalise to four
internal levels and let each adapter translate, accepting that the translation is approximate.

```ts
type Effort = 'minimal' | 'low' | 'medium' | 'high'

const ANTHROPIC_BUDGET: Record<Effort, number | null> = {
  minimal: null,     // omit the thinking block entirely
  low: 5_000,
  medium: 10_000,
  high: 20_000,
}

export function applyEffort(provider: Provider, body: RequestBody, effort: Effort) {
  switch (provider) {
    case 'anthropic': {
      const budget = ANTHROPIC_BUDGET[effort]
      if (budget) body.thinking = { type: 'enabled', budget_tokens: budget }
      return body
    }
    case 'openai':
      body.reasoning_effort = effort === 'minimal' ? 'low' : effort
      return body
    case 'byteplus':
      body.extra_body = { ...body.extra_body, reasoning_effort: effort === 'high' ? 'hi' : effort }
      return body
    default:
      return body // provider ignores it; not an error
  }
}
```

The `default` branch returning the body untouched is deliberate. A provider that does not support
the control should be a no-op, not a thrown error, or adding a provider breaks every call site.

## Which calls actually need it?

The ones where the model has to work something out. Not the ones where it has to pick from a list.

I set effort per agent role rather than per request, because roles are stable and requests are
not:

- **Routing and classification:** minimal. The output space is a handful of tokens. Deliberation
  adds latency and changes the answer roughly never.
- **Summarising and formatting:** low. Structure matters, invention does not.
- **Planning and multi-step tool use:** medium. Getting the order wrong is expensive downstream.
- **Debugging and analysis:** high. This is what the budget is for.

Tie the level to the role in the same place you choose the model, and it stays consistent as
prompts get edited by people who are not thinking about token budgets.

## Measure it, because intuition is wrong here

I would not have found the classification problem by reading code. Aggregate token counts looked
normal. Per-role counts, once I tagged usage by role, showed a step that produced four words
consuming a large share of the bill.

Route everything through a gateway that records tokens per call with the role attached.
[Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/) does this in front of
multiple providers, which is convenient if you are already routing across several.

The number to watch after you turn the dial down is not cost. It is your quality signal for that
role: retry rate, escalation rate, validation failures. Dropping a role to minimal and watching
escalations climb tells you the reasoning was doing something. Dropping it and seeing nothing move
tells you it was not.

My classification path lost nothing measurable at minimal effort. The debugging path would be
worse at anything below high, and I have not tried to save money there.

## Sources

- [Extended thinking](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking), Anthropic documentation
- [Reasoning models](https://platform.openai.com/docs/guides/reasoning), OpenAI platform documentation
- [AI Gateway](https://developers.cloudflare.com/ai-gateway/), Cloudflare
- [Workers AI](https://developers.cloudflare.com/workers-ai/), Cloudflare