PATCHBOOK SERIES Part of the Patchbook Series →

Chapter 3: Knowledge in the loop

Chapter 2 ended on a sealed loop: nothing enters the machine except what you type and what the model carries from training. This chapter is the first crack in the seal.

The seal matters because what the model carries is less than it looks. It knows everything that was public when it was trained, and nothing else. One part of the gap is time: this month's exchange rate, the API of the library released last week. The bigger part was never public at all. No model, however recent, has read your company's JIRA, your internal dashboards, or the refund policy only your support team can see. The problem is not that the model is out of date. The problem is that your world was never in the training set.

This chapter follows one question all the way through: a customer asks support whether an annual plan can be refunded after two months. The answer sits in a document your company wrote and no model has ever seen. Cracking the seal is not a new science, either. It is the oldest one in the office: look it up. When the new colleague asks a question they cannot answer, you do not rewire the colleague. You slide the relevant page across the desk. This chapter builds the slide.

Knowledge on demand

The industry name for the fix is retrieval-augmented generation, RAG, and it sounds like a machine you buy. The act is smaller. Your code takes the question, searches a pile of your documents for the relevant pages, pastes what it finds into the array, and lets the model answer from what it was handed. The model is not consulting a library. You are handing it pages, and it answers the way it always has: by reading the array.

The machine gains one part:

Knowledge in the loop: documents entering host memory

The search can start almost embarrassingly simple. Loop over your documents and keep the ones that share words with the question. The refund question goes in, and the pages containing "refund" and "annual" surface. That is keyword search: plain code, no infrastructure, fully debuggable. It misses the page that says "money back" instead of "refund", and it does not understand the question. It still does real work, and it is the honest first draft of every retrieval system.

Keyword search retrieves candidates, not answers, so the classic upgrade is to retrieve wide and read narrow. Pull fifty loose matches. Score each one against the question, carefully, and keep the best five. The scoring pass is called re-ranking, and you already know the move from hiring: a quick screen of the whole stack, then a close read of the shortlist.

The search step: keyword search, re-rank, the best five into the array

Search can get fancier. Vector search matches on meaning, so "money back" finds "refund". Graph retrieval follows links between documents, so the policy pulls in the amendment that changed it. Both are upgrades to the one arrow in the diagram, and both live in this book's appendix. The loop does not care which searcher you use. To the machine, every searcher is the same arrow.

Now the flaw. The model trusts the pages you hand it, because the array is the conversation: what is in it is, as far as the model can tell, what is true. Retrieve a stale policy page and the model answers with yesterday's rules, in the same confident tone it uses for today's. Nothing at the door checks a document's age, its author, or its honesty. The fix is old-fashioned: curate your sources, and screen what you inject.

Include it whole

There is a brute-force alternative that skips the search entirely: put the whole document in. The refund policy is two pages, so attach the PDF, ask the question, and skip retrieval altogether. This is retrieval with the search left out, and for a document this size it is the right answer. Under the hood it is the array again: the host unpacks the file into blocks, and they enter through the same door as the screenshot from chapter 2. Remember the door.

Attachments work this way: PDF, XML, XLSX. The model does not open the file later, and there is no shelf. The contents land in the array, and the model reads them exactly the way it reads your messages.

And some knowledge was never text at all. The refund policy's 2019 amendment is a scan of a signed memo, and no keyword search can find a clause inside a picture. The same goes for the whiteboard photo from the planning meeting, or the chart in the quarterly deck. A picture rides in as an image block, and the model reads it with the same attention it gives the conversation. This is the other reason the block door mattered.

Include it whole: the forty pages land in the array, the clause one line among many

The flaw is the flood, and it arrives the moment the document grows. Attach the forty-page master agreement instead of the two-page policy, and the refund clause competes with thirty-nine pages of definitions and recitals. The model reads everything you hand it, so every page competes with the one line that matters, and an answer can drown in its own documentation. The bill compounds too: readers of the first book met the context window in chapter 6, and every page you attach is paid for again on every turn. When a document is small, include it whole. When it is not, send the model the part it needs, not the binder it came in.

Try it

The problem with this

Look at the diagram again. The machine changed shape this chapter. Knowledge flows in from above, and the agent can now quote your policies, read your spreadsheets, and cite your code. It still cannot do anything about any of it. Every word it produces leaves through the exit it has always had: the display.

Ask it to fix the bug and it writes you instructions. Ask it to update the spreadsheet and it describes the steps, politely, for you to perform. The knowledge is in the loop. The hands are not. The next chapter is about the first way to act.