Chapter 2: The simplest agent

Chapter 1 ended with a promise: the machine is one loop, and you could write it in an afternoon. This chapter is that afternoon. The whole machine is one list, one call, and a loop around them both, and to watch it run, we will follow one request all the way through: you ask your agent to add rate limiting to the login endpoint, and we stay with that request from your keyboard to the model and back.

The loop

Send the rate limiting request, and five things happen, in order. Your sentence is added to a list. The list is sent to the model. The model writes a reply. The reply is added to the same list. The reply appears on your screen.

That is the machine. In this book's vocabulary: your program is the host, the list inside it is the memory, the call is the API request to the LLM, and the going-around is the loop.

The memory has a precise shape. It is an array of messages, and each message is one line of the conversation so far:

[
  { "role": "user", "content": "add rate limiting to the login endpoint" },
  { "role": "assistant", "content": "I'll start by looking at the login route." }
]

And the loop around it fits in one breath:

memory = []
loop:
    append the user's input to memory
    response = call the LLM with memory
    append the response to memory
    display the response

Readers of the first book met this skeleton as "an agent is a for loop around a stateless engine." Same skeleton. It was the thesis of that book, and it is the floor plan of this one.

Now the lesson under the lesson. The array is not a record of the conversation. The array is the conversation. The model keeps no diary, holds no impression of you, and carries nothing between requests. On every call, it meets the transcript fresh and plays its part to the end. Start a new chat, and the amnesia is total, because the new chat has a new array.

You have felt this without naming it. Edit an old message in a chat, and the model forgets everything that came after it. Nothing was erased from a mind. You rewrote the array, and the model answered the transcript it was handed.

Every loop has a flaw hiding in plain sight, and this one's is append. The loop only ever adds. Every turn makes the array longer, and nothing ever removes anything. A long conversation gets expensive, then it overflows. The fix, trimming and compacting, is the subject of chapter 6. For now, just notice the flaw. It never goes away. It only gets managed.

Everything is a content block

Paste a screenshot of the login route failing under load, with a question attached. You think of that as one message with an attachment. The machine sees something more interesting: one message whose content is an array of two typed parts.

{
  "role": "user",
  "content": [
    { "type": "text", "text": "what is wrong with this login route?" },
    { "type": "image", "source": "..." }
  ]
}

The content of a message is not a string. It is an array of blocks, and each block has a type: text, image, file. Text is simply the most common block. Your screenshot and your question travel together, as two parts of one message, in one slot of the array.

This matters twice. Today, it explains how anything other than words gets into a conversation: it rides in as a block. Later, it explains how tool results get in. When an agent runs a tool and feeds the answer back, the answer comes home as a block in the array. Same door. Remember the door.

The array is not free storage, though. Every block costs tokens, and an image costs far more than the same idea in words. The whole array is re-sent on every turn, so everything in it is paid for again and again. The fix is blunt: include only what the model needs.

Streaming

The rate limiting fix is on its way back, and you can tell, because the reply is typing itself out, word by word. It looks like theater, a trick to make the machine feel alive. The truth is better. The reply is being written right now, in order, one small piece at a time, and the screen is showing you the writing as it happens.

That is why streaming exists as a feature. A long reply takes tens of seconds to produce. Waiting for it in silence feels broken. Watching it arrive feels alive, and for once the feeling is honest: something is actually being written, right now, and you are reading over its shoulder.

Streaming also hands the loop its first taste of partial information. A half-streamed reply is a half-written sentence. Read it too early, and you act on something the model had not finished saying. The rule that follows the loop owner around for the rest of this book: treat a stream as unfinished until the end signal arrives, and validate before acting on any of it.

Try it

• New chats are new arrays. Tell a model a made-up word in one chat. Then open a different chat and ask for the word. Nothing comes back. The array was the memory, and the new chat has a different array.
• One message, two parts. Paste a screenshot into a chat, with a question about it attached. The answer covers the picture and the words together, because both arrived as two blocks of one message.
• Watch writing happen. Ask a model a question that needs a long answer. The words arrive in order because they are being written in order. Nothing is being fetched from storage.

The problem with this

Look at the diagram again, and notice what it does not have. There is no connection to the outside world. Nothing enters this machine except what you type and what the model carries from training. The loop is sealed.

Ask your coding agent about your company's internal dashboard. It will guess, refuse, or invent something plausible. It is not being dumb. The array contains only what you put in it, the model's knowledge stopped at its training date, and your private data was never in the training set.

The next chapter is about the first crack in the seal: how knowledge gets into the loop without you typing it in yourself.