Route cheap models to orchestration, not to the work

The first version of my agent harness used one model for everything. It was the obvious choice: one provider, one set of prompts, one place to tune. It also meant a frontier model was deciding which sub-agent should handle a request, reading a tool manifest, and emitting about forty tokens of JSON. That decision cost the same per token as the reasoning we actually cared about.

Splitting model selection by role rather than by task is where the cost curve bent.

Why is orchestration a routing problem, not a reasoning problem?

An orchestrator agent does a narrow job. It reads a user message, looks at a list of available specialists, and emits a choice. There is very little latent reasoning in that step, and the output space is tiny. Once the tool manifest is well described, a small model is close to indistinguishable from a large one at it.

The mapping lives in one file. Model identifiers are the real ones: Workers AI models are prefixed @cf/, and Anthropic publishes its model identifiers separately.

const ROLE_MODELS = {
  orchestrator: '@cf/meta/llama-3.1-8b-instruct',
  classifier: '@cf/meta/llama-3.1-8b-instruct',
  summariser: 'claude-haiku-4-5',
  reasoner: 'claude-opus-5',
} as const

type Role = keyof typeof ROLE_MODELS

export async function runRole(env: Env, role: Role, prompt: string) {
  const model = ROLE_MODELS[role]

  // Workers AI models run on the binding; frontier models go out over the API.
  if (model.startsWith('@cf/')) {
    return env.AI.run(model, { prompt })
  }

  return callAnthropic(env, model, prompt)
}

The Workers AI binding is declared in the Wrangler config, so no key management is needed for the cheap tier:

[ai]
binding = "AI"

The important property is that the mapping is declarative and lives in one place. When a provider ships something cheaper, you change a string. You do not go hunting for the seven call sites where a model name was inlined.

Measure per role before you optimise

I only found the orchestration overhead because token metering was tagged by role from the start. Aggregate token counts told me I was spending a lot. Per-role counts told me that a third of spend was going to a step whose entire output was a sub-agent name.

Role	Share of calls	Share of spend before	After
Orchestrator	41%	33%	4%
Reasoner	22%	48%	61%
Summariser	37%	19%	35%

Spend did not just move around. Total cost per completed task dropped by a bit over half, and the reasoner got a larger share of a smaller bill, which is the direction you want.

Before reaching for a cheaper model, check what you are re-sending. Anthropic’s prompt caching removes a large part of the cost of a stable system prompt, and it applies regardless of which tier you settle on.

Fallback is part of the design, not an afterthought

Cheap models fail more often, and they fail differently. A small orchestrator will occasionally emit a sub-agent name that does not exist. That is fine, as long as the harness treats an invalid choice as a signal rather than an exception:

1. Validate the choice against the live manifest.
2. On a miss, retry once at the same tier with the manifest restated.
3. On a second miss, escalate to the next tier up and record the escalation.

The escalation rate is the number to watch. If it climbs above a few percent for a given role, that role is mis-tiered and the mapping needs to change. If it sits near zero for weeks, the role is probably over-provisioned and you can try a cheaper tier.

The goal is not the cheapest possible model. It is the cheapest model whose failure mode you have already handled.

What this does not solve

Per-role routing does nothing for a badly shaped task. If a single agent turn is doing retrieval, reasoning and formatting at once, no amount of model selection will make it cheap, because the expensive part is the context you keep re-sending. Decomposition comes first. Routing is what you do once the roles are actually distinct.

Sources

• Workers AI, Cloudflare
• Workers AI models, Cloudflare
• Workers AI binding configuration, Cloudflare
• Claude model identifiers, Anthropic
• Prompt caching, Anthropic