Chapter 3: How to send a chat request to an LLM

Because LLMs are used so often in conversational chatbots, communicating with them feels like sending a text message to a colleague.

Underneath the hood, however, there is no magic. Talking to an LLM is simply sending a standard HTTPS POST request to an API endpoint with a JSON envelope containing a sender and a message.

const botResponse = await sendPost("https://acme-ai.com/v1/chat", {
    sender: "user",
    message: "Hello, World!"
});

The anatomy of a chat request

Every HTTP chat request sent to an LLM provider consists of four essential ingredients:

1. The Endpoint URL: The network address where the provider hosts their inference API (for example, https://api.openai.com/v1/chat/completions or https://acme-ai.com/v1/chat).
2. The Authentication Key: A secret API key passed in the Authorization header (Bearer sk-...) that authenticates your application and bills your account.
3. The Target Model: A string specifying which model weights to run inference against (such as gpt-4o, claude-3-5-sonnet, or llama3).
4. The Message Array: A list of structured message objects detailing the conversation transcript.

When stripped of vendor-specific SDK wrappers, every request envelope boils down to a payload of structured messages.

Message roles explained

Inside the message payload, every message requires a role (or sender) and the text content of that turn.

While different AI providers use slightly different naming conventions, every modern LLM architecture relies on three foundational roles:

[
    { role: "system", content: "You are a helpful TypeScript assistant." },
    { role: "user", content: "How do I reverse an array in place?" },
    { role: "assistant", content: "You can use the Array.prototype.reverse() method." }
]

1. The System Role (system)

The system message acts as the overarching instruction set or rulebook for the model.

It is typically placed at the very beginning of the transcript. It defines who the model is, how it should format its answers, what tone it should adopt, and what safety boundaries it must obey.

{
    role: "system",
    content: "You are an automated code review assistant. Only output concise bullet points highlighting potential bugs."
}

2. The User Role (user)

The user message represents input coming from outside the model.

In a conversational tool, this is the message typed by a human. In a backend pipeline, this can be text scraped from a website, the contents of a customer support email, or diagnostic output from a unit test runner.

{
    role: "user",
    content: "Explain the difference between interface and type in TypeScript."
}

3. The Assistant Role (assistant / bot)

The assistant message represents output previously produced by the model itself.

When starting a brand-new conversation, you rarely send an assistant message in the initial request. But on subsequent turns of your application loop, you must record every answer the model generated and send it back as an assistant entry so the model can see its own prior remarks.

{
    role: "assistant",
    content: "In TypeScript, both interface and type can describe object shapes, but interfaces support declaration merging."
}

The complete API roundtrip

What actually happens across the wire when you make a chat request?

The full roundtrip lifecycle unfolds across four distinct stages:

1. Client Packaging: Your application collects the messages, attaches authentication headers, serializes the payload to JSON, and opens an HTTPS connection to the provider's server.
2. Outbound Dispatch: The payload flies across the internet to the remote server cluster.
3. Inference Execution: The provider's inference cluster feeds the full transcript through neural network layers, token by token, calculating the most probable continuation until a stopping condition is reached.
4. Inbound Response: The server sends back an HTTP 200 OK response enclosing the generated text, completion status, and a token usage receipt:

console.log(botResponse);
// {
//     sender: "bot",
//     message: "Why, hello there! How can I assist you today?",
//     usage: { promptTokens: 12, completionTokens: 11, totalTokens: 23 }
// }

Wire format vs SDK wrappers

Many developers only interact with LLMs through high-level SDKs like openai.chat.completions.create().

While SDKs provide convenient type definitions and automatic retries, it is vital to remember that they are thin wrappers over basic HTTP POST requests. Under the hood, every single provider is listening for a standard JSON envelope with a headers list, a model identifier, and a list of message objects.

In the next chapter, we will look at the most crucial implication of this architecture: why LLMs are completely stateless, and why your host program must serve as the conversation's memory.