# Route cheap models to orchestration, not to the work

Most agent platforms pick one model and use it everywhere. Splitting model selection by role rather than by task is where the cost curve actually bends.

By Elson Tan (https://elsontan.com)
Published: 2026-07-14
Updated: 2026-08-01
Reading time: 4 min
Tags: LLM, Architecture, Cost
Canonical: https://elsontan.com/blog/routing-cheap-models-to-orchestration/

> Assign models by agent role, not by task. Orchestration and classification are routing problems with tiny output spaces, so a small model handles them as well as a frontier one. Reserving frontier models for actual reasoning cut my cost per completed task by just over half.

---
The first version of my agent harness used one model for everything. It was the obvious
choice: one provider, one set of prompts, one place to tune. It also meant a frontier model
was deciding which sub-agent should handle a request, reading a tool manifest, and emitting
about forty tokens of JSON. That decision cost the same per token as the reasoning we
actually cared about.

Splitting model selection by **role** rather than by task is where the cost curve bent.

## Why is orchestration a routing problem, not a reasoning problem?

An orchestrator agent does a narrow job. It reads a user message, looks at a list of
available specialists, and emits a choice. There is very little latent reasoning in that
step, and the output space is tiny. Once the tool manifest is well described, a small model
is close to indistinguishable from a large one at it.

The mapping lives in one file. Model identifiers are the real ones:
[Workers AI](https://developers.cloudflare.com/workers-ai/) models are prefixed `@cf/`, and
Anthropic publishes its
[model identifiers](https://docs.anthropic.com/en/docs/about-claude/models) separately.

```ts
const ROLE_MODELS = {
  orchestrator: '@cf/meta/llama-3.1-8b-instruct',
  classifier: '@cf/meta/llama-3.1-8b-instruct',
  summariser: 'claude-haiku-4-5',
  reasoner: 'claude-opus-5',
} as const

type Role = keyof typeof ROLE_MODELS

export async function runRole(env: Env, role: Role, prompt: string) {
  const model = ROLE_MODELS[role]

  // Workers AI models run on the binding; frontier models go out over the API.
  if (model.startsWith('@cf/')) {
    return env.AI.run(model, { prompt })
  }

  return callAnthropic(env, model, prompt)
}
```

The Workers AI binding is declared in the Wrangler config, so no key management is needed for
the cheap tier:

```toml
[ai]
binding = "AI"
```

The important property is that the mapping is declarative and lives in one place. When a
provider ships something cheaper, you change a string. You do not go hunting for the seven
call sites where a model name was inlined.

## Measure per role before you optimise

I only found the orchestration overhead because token metering was tagged by role from the
start. Aggregate token counts told me I was spending a lot. Per-role counts told me that a
third of spend was going to a step whose entire output was a sub-agent name.

| Role | Share of calls | Share of spend before | After |
| --- | --- | --- | --- |
| Orchestrator | 41% | 33% | 4% |
| Reasoner | 22% | 48% | 61% |
| Summariser | 37% | 19% | 35% |

Spend did not just move around. Total cost per completed task dropped by a bit over half,
and the reasoner got a larger share of a smaller bill, which is the direction you want.

Before reaching for a cheaper model, check what you are re-sending. Anthropic's
[prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) removes
a large part of the cost of a stable system prompt, and it applies regardless of which tier
you settle on.

## Fallback is part of the design, not an afterthought

Cheap models fail more often, and they fail differently. A small orchestrator will
occasionally emit a sub-agent name that does not exist. That is fine, as long as the harness
treats an invalid choice as a signal rather than an exception:

1. Validate the choice against the live manifest.
2. On a miss, retry once at the same tier with the manifest restated.
3. On a second miss, escalate to the next tier up and record the escalation.

The escalation rate is the number to watch. If it climbs above a few percent for a given
role, that role is mis-tiered and the mapping needs to change. If it sits near zero for
weeks, the role is probably over-provisioned and you can try a cheaper tier.

> The goal is not the cheapest possible model. It is the cheapest model whose failure mode
> you have already handled.

## What this does not solve

Per-role routing does nothing for a badly shaped task. If a single agent turn is doing
retrieval, reasoning and formatting at once, no amount of model selection will make it
cheap, because the expensive part is the context you keep re-sending. Decomposition comes
first. Routing is what you do once the roles are actually distinct.

## Sources

- [Workers AI](https://developers.cloudflare.com/workers-ai/), Cloudflare
- [Workers AI models](https://developers.cloudflare.com/workers-ai/models/), Cloudflare
- [Workers AI binding configuration](https://developers.cloudflare.com/workers-ai/get-started/workers-wrangler/), Cloudflare
- [Claude model identifiers](https://docs.anthropic.com/en/docs/about-claude/models), Anthropic
- [Prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching), Anthropic