Persist once, reference by id, hydrate on demand

A user dropped in three vendor bills and asked for a summary. The agent read them and got it right.

Then they said “good, create the draft bills for those” and the agent told them it could not access the files any more.

Ninety seconds. Same conversation. Same window.

It looked like a memory bug and I spent a while treating it as one. It was not. On the upload turn I had pasted the extracted text straight into the prompt, then thrown it away, leaving a marker like [upload_ref:8f2a...] in the history. The agent could see that marker on every later turn and had absolutely no way to turn it back into a document.

It was not forgetting. It had never been given anything to look at.

Both obvious fixes are wrong

Once you see the bug, two fixes come to mind, and neither works.

Keep the file text in the conversation history, and you pay for those three PDFs on every single turn afterwards. Tens of thousands of tokens, resent when the user asks something completely unrelated.

Drop it after the turn, which is what I had done, and you get the amnesia.

There is no setting that fixes this, because the file is simply in the wrong place.

The shape that works

Persist once, reference by id, hydrate on demand.

Full text goes to object storage at upload time, keyed by a stable id. History carries only a short marker and a one-line synopsis, enough for the model to know the file exists and roughly what is in it. When a turn actually needs the content, the model calls a tool and pulls it back.

// At upload: extract once, persist, return a compact handle.
const id = crypto.randomUUID().slice(0, 8)
await env.FILES.put(`${sessionId}/${id}`, extractedText)

this.sql.exec(
  `INSERT INTO session_file (id, session_id, filename, synopsis, bytes, created_at)
   VALUES (?, ?, ?, ?, ?, ?)`,
  id, sessionId, file.name, synopsis, extractedText.length, Date.now()
)

// What history sees, on this turn and every later one:
// [[file:a3f9c210]] invoice-vendor-3.pdf, 2 pages, supplier invoice dated 2026-03-14

The tool the model calls is deliberately boring:

{
  name: 'read_session_file',
  description: 'Read the full text of a file the user uploaded in this session.',
  input_schema: {
    type: 'object',
    properties: {
      file_id: { type: 'string', description: 'The id inside [[file:...]] in the conversation' },
      offset:  { type: 'integer' },
      limit:   { type: 'integer', description: 'Characters to return, default 20000' },
    },
    required: ['file_id'],
  },
}

Offset and limit matter more than they look. Without them a single tool call on a large document puts you straight back into the problem you were solving, except now it happens mid-turn where truncation is harder to see.

This is the same pattern Anthropic’s Files API implements: upload once, reference by id across requests, rather than resending bytes.

What does this actually save?

Turns that do not touch the file pay nothing for it.

That is the whole economic argument, and it is worth being precise about it rather than quoting a percentage. If a conversation runs twelve turns and two of them involve the uploaded document, the inline approach pays for the document twelve times and this approach pays twice, plus about twenty tokens per turn for the marker and synopsis.

The synopsis is what makes the model behave. Given a bare id it does not know whether reading is worth a tool call. Given “supplier invoice dated 2026-03-14” it can answer “which of these is from March” without reading anything at all.

Worth pairing with prompt caching, which addresses the other half of the growth curve: the stable system prompt and tool definitions that get resent every turn regardless of what you do with files.

Sub-agents need the id, not the text

The failure I did not anticipate: an orchestrator that delegates to a sub-agent and helpfully pastes the file content into the delegation prompt. That reintroduces the cost, and worse, the sub-agent’s copy can drift from the stored one if the file is ever re-extracted.

The delegation payload carries ids. The sub-agent gets the same read_session_file tool and resolves them itself. One source of truth, and the orchestrator never handles the bytes.

Lifecycle, which is on you

Storage keyed by session grows until something deletes it. I expire session files on a fixed window after last access and let the tool return a clear “this file has expired” rather than an empty string, because an empty string makes the model hallucinate content it thinks should be there.

Object storage with a lifecycle policy is the natural home for this. R2 is what I use, alongside the metadata table in the session’s own Durable Object so the index lives with the conversation it belongs to.

Sources

• Files API, Anthropic documentation
• Prompt caching, Anthropic documentation
• Cloudflare R2, Cloudflare
• Durable Objects SQL storage API, Cloudflare