Elson TanA teacher tried our study assistant, got an answer about photosynthesis, and asked a fair question: where does that come from? The app said “Source: Form 4 Science”. She had the book on her desk. She still could not find the sentence.
That is the whole problem. She did not need the app to be right. She needed to be able to check it in ten seconds, and we had given her a book title and 300 pages.
So my team and I built the thing backwards from her. Every answer has to open the textbook at the page it came from. That one rule threw out most of the designs we had been considering, including the obvious one: asking the model to write its own citations. A model that invents the prose will happily invent the page number too.
Attach the page number when you chunk, or lose it forever
The page number has to ride along with the text from the moment you split the book up.
You might think you can work it out later by matching the answer against the source. You cannot. The model paraphrases, so the words in the answer are not the words on the page, and the matching fails hardest on exactly the answers a teacher most wants to check.
So the chunker records where each piece came from:
chunks.append({
"id": f"{doc_id}:{page_num}:{idx}",
"text": chunk_text,
"metadata": {
"doc_id": doc_id,
"subject": subject,
"page": page_num, # 1-based, matches the printed page
"chunk_index": idx,
},
})Every vector database worth using stores metadata beside the embedding and returns it on query. Chroma does, which is what we index into.
The chunk id encodes document, page and position, so a citation is resolvable without a second lookup. That is a small decision that removes a join from the hot path.
Make the model cite the id, not the source
The prompt puts the retrieved chunks in with their ids visible, and requires the answer to reference them:
Answer using only the passages below. After each claim, cite the passage id
in square brackets, for example [sci4:112:2]. If the passages do not contain
the answer, say so rather than answering from general knowledge.
[sci4:112:2] (page 112)
Photosynthesis converts light energy into chemical energy stored as glucose...
[sci4:113:0] (page 113)
The light-dependent reactions occur in the thylakoid membrane...I ask for the id and not the page number on purpose. An id can be checked. Either it matches a chunk I actually put in the context or it does not, and anything that does not match gets stripped out before the answer reaches the screen.
A page number cannot be checked that way. It is just a number, and a model will happily produce a believable one.
| What you ask the model to cite | Verifiable after generation | What a wrong one looks like |
|---|---|---|
| Document name | No | Correct, and useless: 300 pages to search |
| Page number | No | A plausible number pointing at the wrong page |
| Chunk id | Yes, against what you retrieved | Fails the lookup and gets stripped before display |
| Quoted sentence | Partly, by string match | Paraphrase defeats the match, which is when you need it most |
The check is about four lines of code. It is also the only thing standing between citations that mean something and citations that merely look like they do.
$cited = collect($matches)->unique()
->filter(fn ($id) => isset($retrievedById[$id]));What happens when nothing relevant comes back?
The model says so. This is a retrieval configuration question as much as a prompt one, and it is the failure mode with the worst consequences in an education product, because a confident wrong answer to a syllabus question is actively harmful.
Two controls. A similarity floor, below which a chunk is not passed to the model at all, so a question about an unindexed topic arrives with an empty context. And an explicit instruction for the empty case, because a model given no passages and no instruction will answer from its pretraining and sound exactly as confident.
Routing helps here too. An orchestrator detects the subject first and queries only that subject’s collection, which keeps a Bahasa Malaysia question from retrieving loosely similar Science passages.
| Control | Stops | Without it |
|---|---|---|
| Similarity floor | Weak matches reaching the prompt | Loosely related passages arrive and look authoritative |
| Explicit empty-context instruction | Answering from pretraining | The model answers anyway, at the same confidence |
| Subject routing before retrieval | Cross-subject matches | A language question retrieves science passages that scored well |
| Id validation after generation | Invented citations | Everything looks right in testing and fails on a real question |
Link to the page, not the document
The last step is the one users notice. A citation resolves to a link that opens the source at the cited page.
For a PDF in a viewer that supports it, that is a fragment: #page=112. The textbooks are not
public, so the link is a time-limited signed URL generated per request, the same mechanism as
R2 presigned URLs or the equivalent
on other object stores. Access control stays with the storage layer instead of being reimplemented
in the app.
| Layer | Carries | Failure if missing |
|---|---|---|
| Chunking | Page number, chunk id | Citations cannot be resolved at all |
| Vector metadata | Same, returned on query | Retrieval loses provenance |
| Prompt | Ids visible to the model | Model cites something ungrounded |
| Post-processing | Id validation | Invented citations reach the user |
| Rendering | Page-anchored signed link | Teacher cannot check the claim |
Every row has to hold. Skipping the fourth is the common one, because everything still looks right in testing.
Sources
- Chroma documentation, Chroma
- Gemini API documentation, Google
- R2 presigned URLs, Cloudflare