Request anatomy

What you will be able to do

Read every part of a real request to a language model and of its response, and know what each part does. Then a setting that the model rejects, or an answer that stops halfway, does not surprise your code.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The parts of an HTTP request (method, path, headers, body) and status codes, from HTTP fundamentals in Build and Deploy Your First API.
• Temperature, from Predicting continuations: Temperature, and hidden reasoning tokens, from Reasoning and multimodality: The idea. Both are in LLMs: How They Work and How to Choose One.

What you need on your computer

• The project ticket-assistant as you left it in A first run without an account, with .venv active: python -m pytest gives 2 passed. If it does not, reset the project as in that lesson.

You need no account and no key. The last section is an optional live call: with Ollama it is free; with your own key it costs a fraction of a cent.

The form at the courier’s counter

To send a parcel, you fill in a form. It has fixed boxes: the address, your account number, the service (standard or express), the contents, and the most that you will pay. The receipt also has fixed boxes: a tracking number, the weight, the price, and later a status, such as “delivered” or “returned: address incomplete”.

You do not need to know how the courier’s trucks work. But you must know every box on the form. A wrong box gives a parcel at the wrong address, or one that never leaves.

The problem: a field you do not understand fails quietly

AmiraTermAmiraLarkfield’s developer. She writes the code of the help-desk tools in the courses. wanted the assistant to give the same answer to the same ticket, every time. In the language models course, a temperatureTermTemperatureA setting that controls how much a model varies its choice of the next token. At 0, it almost always takes the most likely token. Higher values give more varied answers.Learn more Predicting continuationsLLMs: How They Work and How to Choose One of 0 did that. So she added "temperature": 0 to the request. The provider answered with an error, HTTP 400:

Unsupported value: 'temperature' does not support 0 with this model. Only the default (1) value is supported.

Then she wanted to save money, and set the output limit to 40 tokens. This time there was no error at all. The response had status 200, and the answer was empty: "". Both are real responses of chat-smallTermchat-smallThe name that Larkfield’s code gives to a small, low-cost hosted model: gpt-6-luna on Azure. The code uses the name, so the model can change without a code change.Learn more What an LLM API isBuilding Reliable Applications with LLM APIs.

The first failure is loud: the code sees an error. The second is quiet: code that reads only the answer text gets nothing and does not know why. To avoid both, you must read the request and the response field by field.

The idea: every field has one job

The anatomy of a request is the list of its parts and the job of each, as the anatomy of a body lists its organs. Here are the parts of a Chat Completions request:

• The endpoint: the address of the provider and the path of one operation, here /chat/completions.
• The credential: the API key, in the Authorization header.
• The model identifier: which model must answer.
• The messages, each with a role. system holds the application’s instructions. user holds the text to work on. assistant holds an earlier answer of the model, in a conversation (Module 5). Some providers name the system role developer.
• The generation settings: for example max_completion_tokens, the output limit, and temperature.

The response has its own parts: an HTTP status and headers, an ID and the model that answered. It also has the answer, as a message with the role assistant, the finish reason and the usage. The finish reason says why the model stopped: "stop" when it finished, "length" when it reached the output limit.

A tiny worked example: the real request for T-80008

Here is the real request, as plain HTTP. It was recorded for ticket T-80008, prompt v1 and chat-small. The key and the resource name are hidden. The system message has 4,407 characters, so most of it is cut here.

POST https://<your-resource>.cognitiveservices.azure.com/openai/v1/chat/completions
Authorization: Bearer [REDACTED]
Content-Type: application/json

{
  "model": "chat-small",
  "messages": [
    {"role": "system", "content": "You are Larkfield's support assistant. Read the support policy and the customer's ticket. … Policy: …"},
    {"role": "user", "content": "Ticket:\nHello, where is my order LK-581106? The confirmation email said the garden shed shipped on Monday, but I have no delivery date yet."}
  ],
  "max_completion_tokens": 1000
}

And the response, 3.44 seconds later. Azure also sends content-filter reports and timing details; they are left out here.

HTTP/1.1 200 OK
x-request-id: 62aebed5-b92d-497b-9460-65b9c49be4e8
x-ratelimit-limit-tokens: 100000
x-ratelimit-remaining-tokens: 98968

{
  "id": "chatcmpl-EWuZQpxY7DZbRE4aO70zlg2uVrBCj",
  "model": "gpt-6-luna-2026-09-22",
  "choices": [{
    "message": {"role": "assistant", "content": "{\"language\":\"en\",\"team\":\"delivery\", … }", "refusal": null},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 1032, "prompt_tokens_details": {"cached_tokens": 1029},
    "completion_tokens": 308, "completion_tokens_details": {"reasoning_tokens": 187},
    "total_tokens": 1340
  }
}

Read the numbers. 1,032 tokens went in. The provider’s cache already held 1,029 of them, because the same instructions came first in many earlier requests. 308 tokens came out, and 187 of those were hidden reasoning: chat-small is a reasoning modelTermReasoning modelA model that writes hidden reasoning tokens before its answer. chat-small is one. Such a model often accepts only its default temperature, and its hidden tokens count against the output limit.Learn more Reasoning and multimodalityLLMs: How They Work and How to Choose One. You pay for them as output, but you never see them. The rate-limit header shows the budget for the current window: 100,000 − 1,032 = 98,968 tokens left.

The diagram follows the same call through the project’s code, from the help-desk app to the provider and back.

One real call for T-80008,: who sends what to whom. Only the adapter talks to the provider.

Text description of the diagram

Four participants, from left to right: the help-desk app, the assistant, the adapter and the provider. The assistant is the files main.py and context.py, the adapter is providers.py, and the provider is chat-small on Azure. The app gives ticket T-80008 to the assistant. The assistant builds the request: model chat-small, a system message and a user message, max_completion_tokens 1000. It asks the adapter to complete the request. The adapter sends POST /openai/v1/chat/completions to the provider, with the key in the Authorization header. The model writes 308 tokens, 187 of them hidden reasoning. The provider answers HTTP 200 after 3.44 seconds, with request ID 62aebed5…. The answer has the content, finish_reason “stop” and the usage: 1,032 tokens in and 308 out. The adapter turns this into a Completion: the text, the finish reason and the token counts. The assistant gives the app the team (delivery), needs_human (false) and a reply draft. An agent reads the draft before anything is sent.

Explore one field at a time

The explorer holds the same request and response. Pick a field: the explorer marks its lines, says what it does, and shows a real variation. Each variation is the same request with that one field changed, sent on the same evening, with the provider’s real answer.

Explore

One real request and response, field by field

Field

model

Where

The request

Real variations

1

The model identifier: which model must answer. On Azure it is the name of a deployment (chat-small), which points to one model version. Most other providers take the model name itself.

The recorded request (T-80008). The marked lines are model.

POST https://<your-resource>.cognitiveservices.azure.com/openai/v1/chat/completions
Authorization: Bearer [REDACTED]
Content-Type: application/json
 
{
  "model": "chat-small",
  "messages": [
    {
      "role": "system",
      "content": "You are Larkfield's support assistant. Read the support policy and the customer's ticket. … (4,318 more characters: the seven fields to return and Grace's policy)"
    },
    {
      "role": "user",
      "content": "Ticket:\nHello, where is my order LK-581106? The confirmation email said the garden shed shipped on Monday, but I have no delivery date yet."
    }
  ],
  "max_completion_tokens": 1000
}
 
(not sent: temperature, so the model uses its default)

A real variation: one change

Change: "model": "chat-strong" (a larger model, gpt-6.1-sol)

• Status: 200
• finish_reason: stop
• Tokens: 1,032 in (0 cached), 150 out (0 reasoning)
• Model: gpt-6.1-sol-2026-09-29, 3.45 s

The content:

{
  "language": "en",
  "team": "delivery",
  "needs_human": false,
  "reason": "The customer is asking for a delivery update on their own shipped order, with no concerns requiring human handling.",
  "confidence": "high",
  "order": {
    "order_id": "LK-581106",
    "status": null
  },
  "reply": "Hello, standard delivery in Canada takes 3 to 5 business days. Please check your shipping confirmation email for a tracking link, which may show an estimated delivery date once the courier updates it. If no tracking link is available, let us know so the delivery team can check order LK-581106."
}

chat-strong (gpt-6.1-sol) gave the same team and needs_human with 150 output tokens and no reasoning tokens. Its price is US$2 / 10 per million tokens, so this one call cost about 14 times as much as chat-small.

Real calls to gpt-6-luna on Azure (chat-small), except the endpoint variation (Gemma 3 4B through Ollama). The key and the resource name are hidden. Azure also returns content-filter reports and timing details; they are left out here.

Ticket T-80008, prompt v1, chat-small (gpt-6-luna on Azure), with one real variation per field.

Text description of the explorer

The explorer has two groups of buttons. In the request: endpoint, Authorization, model, role system, role user, temperature, max_completion_tokens. In the response: status and headers, id, model (returned), role assistant, finish_reason, usage. For the chosen field, it marks the field’s lines in the request or the response, explains the field, and shows its real variations.

The variations, all:

• Endpoint: the same messages sent to OllamaTermOllamaA free program that downloads open models and runs them on your own computer. Your code calls it at a local address, as it calls a hosted API.Learn more Hosted and self-hosted choicesLLMs: How They Work and How to Choose One on a laptop with Gemma 3 4BTermGemma 3 4BA small open model from Google, with 4 billion parameters. It can run on a laptop with Ollama, without an account or a cost per call.Learn more Hosted and self-hosted choicesLLMs: How They Work and How to Choose One. HTTP 200; the answer is JSON inside a MarkdownTermMarkdownA plain-text way to show formatting, for example **bold** or # for a heading. Models often add it, so a program that expects plain JSON must remove it or reject the answer. fence, with needs_human true; 1,092 tokens in, 189 out.
• Authorization: a wrong key. HTTP 401, “Access denied due to invalid subscription key or wrong API endpoint.” No tokens used.
• Model: chat-strong (gpt-6.1-sol). HTTP 200, the same team and needs_human, 1,032 tokens in, 150 out, no reasoning tokensTermReasoning tokensTokens that some models write before the answer, to work through the problem. You do not see them, but they count against the output limit and you pay for them.Learn more Reasoning and multimodalityLLMs: How They Work and How to Choose One. It cost about US$0.0036, about 14 times as much as chat-small.
• Role system: the system message removed. HTTP 200, finish_reason stop, a plain-text reply that tells the customer to contact “the retailer”; no JSON; 38 tokens in, 257 out.
• Temperature: 0 added. HTTP 400, “Unsupported value: ‘temperature’ does not support 0 with this model. Only the default (1) value is supported.”
• max_completion_tokens: 40. HTTP 200, finish_reason length, empty content, all 40 output tokens were reasoning. And 60 with reasoning turned off: finish_reason length, the JSON cut after the order field, not valid JSON.
• id and role assistant: the same request sent again. A new ID and a different reply, with the same team and needs_human; 238 tokens out.
• finish_reason: the limit of 40 tokens gives “length”.
• Usage: chat-strong used 150 output tokens and no reasoning tokens.
• Role user, status and headers, and the returned model have no variation.

Settings do not travel between models

The temperature error teaches the most general lesson of this page. A request that works on one model can fail on another. chat-small is a reasoning model, and it accepts only its default temperature. The language models course ran Gemma 3 4B at temperatures 0 and 0.8 without a problem. The name of the output limit differs too. The current name is max_completion_tokens; the older name is max_tokens. In a test, Ollama ignored max_completion_tokens: with a limit of 30, Gemma 3 4B still wrote 172 tokens. It obeyed max_tokens. The project sends max_completion_tokens, so on the local route, the limit has no effect.

So for every model that you use, read the provider’s documentation for the settings, and test one real request before you build on it. Module 4 meets a bigger difference: one of the two hosted modelsTermHosted modelA model that runs on a provider’s computers. Your code sends it requests over the internet, and you pay for each call.Learn more Hosted and self-hosted choicesLLMs: How They Work and How to Choose One cannot use tools at all.

Step 1: see the request that the project sends

You now add a small tool to the project: assistant/raw.py. It prints the exact request body that the assistant would send. With --send, it sends the body with httpx, a plain HTTP library, and prints the raw response. It is the project’s window on the wire.

1. Make the file assistant/raw.py with this content:

   assistant/raw.py

   """See one request as JSON, and (optionally) send it with plain HTTP, without the SDK.
   
       python -m assistant.raw T-80008            print the request body that the assistant would send
       python -m assistant.raw T-80008 --send     send it to ASSISTANT_BASE_URL with httpx and print the response
   
   --send needs a live provider (for example Ollama: ASSISTANT_BASE_URL=http://localhost:11434/v1 and
   ASSISTANT_MODEL=gemma3:4b). The key is sent in a header and never printed.
   """
   
   import json
   import os
   import sys
   
   import httpx
   
   from .context import build_request
   from .data import load_ticket
   
   
   def main(argv: list[str]) -> int:
       model = os.environ.get("ASSISTANT_MODEL", "chat-small")
       base_url = os.environ.get("ASSISTANT_BASE_URL", "")
       api_key = os.environ.get("ASSISTANT_API_KEY", "")
       ticket = load_ticket(argv[0] if argv else "T-80008")
       body = build_request(ticket, model, "v1")
       print(json.dumps(body, indent=2, ensure_ascii=False))
       if "--send" not in argv:
           return 0
       if not base_url:
           print("Set ASSISTANT_BASE_URL first, for example http://localhost:11434/v1 for Ollama.", file=sys.stderr)
           return 2
       headers = {"Content-Type": "application/json"}
       if api_key:
           headers["Authorization"] = f"Bearer {api_key}"
       response = httpx.post(base_url.rstrip("/") + "/chat/completions", json=body, headers=headers,
                             timeout=30)
       print(f"\nHTTP {response.status_code}")
       for name in ("content-type", "date", "x-request-id"):
           if name in response.headers:
               print(f"{name}: {response.headers[name]}")
       print(json.dumps(response.json(), indent=2, ensure_ascii=False))
       return 0 if response.is_success else 1
   
   
   if __name__ == "__main__":
       sys.exit(main(sys.argv[1:]))

2. Print the request for T-80008:

   python -m assistant.raw T-80008

   The output starts and ends like this. The long system message is cut here:

   {
     "model": "chat-small",
     "messages": [
       {
         "role": "system",
         "content": "You are Larkfield's support assistant. Read the support policy and the customer's ticket.\n\nAnswer with one JSON object and nothing else, …"
       },
       {
         "role": "user",
         "content": "Ticket:\nHello, where is my order LK-581106? The confirmation email said the garden shed shipped on Monday, but I have no delivery date yet."
       }
     ],
     "max_completion_tokens": 1000
   }

3. Read the system message in your terminal. Find the seven fields that it asks for, and the start of GraceTermGraceThe lead of Larkfield’s help desk. She writes the support policy and decides what a good answer to a customer is.’s policy.

raw.py reads three environment variables: named values that the shell gives to every program that it starts, as in Configuration and secrets in the Python course. They hold the model name, the provider’s address and the key. They are empty now, so the model is chat-small and nothing is sent. The next lesson explains why settings and keys belong in environment variables, not in code.

Step 2: what the SDK adds

raw.py builds the HTTP request by hand. The official openai Python package, the SDK (software development kit), does the same work for you. It also turns the JSON of the response into Python objects with named fields. Many providers and local servers speak the same Chat Completions format, so the same SDK works with all of them.

The SDK is a convenience over plain HTTP. raw.py skips it, so you can see the JSON on the wire.

Text description of the diagram

The project’s build_request makes a Python dictionary with the model, the messages and max_completion_tokens. The openai SDK takes it in client.chat.completions.create(**request). Under the SDK, the httpx library sends a POST over HTTPS to the provider’s endpoint. The POST has the JSON body and the headers, with the key. raw.py skips the SDK: it calls httpx.post with the URL and the body directly. Both routes send the same request.

You can see what the SDK does without a live call: give it a recorded response. Start Python in the project folder with python, and type these lines:

import json
from openai.types.chat import ChatCompletion
lines = open("recordings/eval_text.jsonl", encoding="utf-8").read().splitlines()
entry = [json.loads(x) for x in lines if '"3b337102809abc06"' in x][0]
response = ChatCompletion.model_validate(entry["response"])
response.model
response.choices[0].finish_reason
response.usage.prompt_tokens, response.usage.completion_tokens
response.usage.completion_tokens_details.reasoning_tokens

The output:

'gpt-6-luna-2026-09-22'
'stop'
(1032, 241)
115

3b337102809abc06 is the request key of T-80008 from the last lesson. The SDK checked the JSON against its own model of a response and gave each field a name. This recording used 241 output tokens, 115 of them reasoning: it is a different call from the one in the explorer. Type exit() to leave Python. The next lesson puts the SDK behind one class in the project.

When to use which: the SDK for the application, because it handles the details of HTTP and gives typed objects. Plain HTTP to learn, to debug, and to see exactly what travels.

Guided practice: read the explorer

1. Make your prediction in the explorer, then select I have made my prediction.
2. Select temperature. You should see HTTP 400 and “Only the default (1) value is supported”.
3. Select max_completion_tokens. In the first variation, find finish_reason and the number of reasoning tokens.
4. Select role: system. Read the reply. What is missing, compared with the original answer?
5. Select usage. Compute the cost of the original call and of the chat-strong call. Use US$0.10 / 0.50 per million tokens for chat-small and US$2 / 10 for chat-strong.

Show the answers

2. The provider refused the request before the model ran. chat-small is a reasoning model and accepts only its default temperature. Amira cannot make its answers repeatable with this setting.
3. finish_reason: "length", and all 40 output tokens were reasoning tokens: the content is empty.
4. Everything that the application needs: no JSON, no team, no needs_human. The model answered the customer directly and sent them to “the retailer”, because it did not know that it works for LarkfieldTermLarkfieldAn online shop for the home. It is not a real company. All Nextia courses use its help desk and data as the example.Learn more AI in everyday productsAI Career Foundations: Roles, Skills and Roadmaps.
5. For chat-small: 1,032 × 0.10 + 308 × 0.50 = 257 millionths of a dollar, about US$0.00026. For chat-strong: 1,032 × 2 + 150 × 10 = 3,564 millionths, about US$0.0036. So chat-strong costs about 14 times as much for this easy ticket.

Your turn: one change, one prediction

Ticket T-80001 is about a neighbour’s order. Print its request with python -m assistant.raw T-80001. Before you run it, predict: which fields of the request differ from T-80008, and which stay the same?

Show the answer

Only the content of the user message changes: it holds Tom’s ticket instead. The model, the system message (the same instructions and policy) and max_completion_tokens stay the same. So the first 1,000 or so tokens of every request are identical. That is why the provider’s cache could hold 1,029 of the 1,032 input tokens. The response for T-80001 in the recordings used 1,054 input tokens, because Tom’s ticket is longer.

Optional: a live call

Everything above works with recordings. If you want to see a live answer, you have two routes. Skip this section if you have neither: no later lesson needs it.

Route 1: a free local model with Ollama. Ollama runs open models on your computer and offers the same Chat Completions endpoint. Gemma 3 4B needs about 3.3 GB of disk. The course ran it with Ollama 0.40.1 on a Mac mini with 16 GB of memory.

1. Install Ollama from its website, and start it.

2. Download the model, once:

   ollama pull gemma3:4b

3. Send the request:

   macOS

   ASSISTANT_BASE_URL=http://localhost:11434/v1 ASSISTANT_MODEL=gemma3:4b python -m assistant.raw T-80008 --send

   Windows

   $env:ASSISTANT_BASE_URL = "http://localhost:11434/v1"
   $env:ASSISTANT_MODEL = "gemma3:4b"
   python -m assistant.raw T-80008 --send

   Linux

   ASSISTANT_BASE_URL=http://localhost:11434/v1 ASSISTANT_MODEL=gemma3:4b python -m assistant.raw T-80008 --send

   On macOS and Linux, the variables hold for this one command only. In PowerShell, they hold until you close the window.

After the request, the output shows the response. This is the end of a real run (gemma3:4b, Ollama 0.40.1). The content is cut here:

HTTP 200
content-type: application/json
date: Fri, 09 Oct 2026 02:52:58 GMT
{
  "id": "chatcmpl-244",
  "object": "chat.completion",
  "created": 1791514378,
  "model": "gemma3:4b",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "```json\n{\n  \"language\": \"en\",\n  \"team\": \"delivery\",\n  \"needs_human\": true, … \"reply\": \"Hello, I understand you're checking on the status of order LK-581106, which was shipped on Monday. I’ve checked the tracking information and it currently shows as ‘In Transit’ with an estimated delivery date of Wednesday. …\"\n}\n```"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 1092,
    "prompt_tokens_details": {
      "cached_tokens": 1091
    },
    "completion_tokens": 191,
    "total_tokens": 1283
  }
}

The same fields, from another provider. Two differences matter. The usage has no reasoning tokens: Gemma 3 4B does not reason. And the reply says “I’ve checked the tracking information… estimated delivery date of Wednesday”. The model has no tracking data: it invented this. Your run gives a different reply.

Route 2: your own key with a hosted provider. Any provider with an OpenAI-compatible Chat Completions endpoint works. Set three variables: ASSISTANT_BASE_URL (for OpenAI, https://api.openai.com/v1), ASSISTANT_MODEL (a model name from your provider’s list) and ASSISTANT_API_KEY (your key). One request for T-80008 sends about 1,000 input tokens and gets a few hundred back. With a small model, that costs a fraction of a cent. The course tested this route with Azure’s OpenAI-compatible endpoint, not with OpenAI itself.

Type the key only in your terminal, never in a file of the project. The next lesson shows the safe place for it.

Common mistake

An empty answer, and no error

Symptom: Amira lowers the output limit to 40 tokens to save money. The provider answers with status 200, and the assistant shows an empty reply. Nothing in the logs says that something went wrong.

Cause: chat-small reasons before it writes. All 40 tokens went to hidden reasoning, so the answer text is empty. The response said so with finish_reason: "length" and reasoning_tokens: 40. The code read only the content.

Fix: read finish_reason on every response. "length" means that the answer is cut or missing: never use it as a finished answer. Give a reasoning model room: the project uses 1,000 tokens. With this prompt, chat-small used up to 600 reasoning tokens on one of the course’s tickets. Module 3 turns this check into code.

What it means for a real AI service: a status of 200 means that the provider did its job. It does not mean that you have a usable answer. The finish reason and the usage tell you whether you do.

Knowledge check

Check your understanding

Name the job of each part: the endpoint, the Authorization header, the system message, the user message and max_completion_tokens.

Show the answer

The endpoint is the address of the provider and the operation (POST /chat/completions). The Authorization header carries the API key, so the provider knows who sends the request and who pays. The system message holds the application’s instructions. The user message holds the text to work on, here the ticket. max_completion_tokens is the most tokens that the model may produce, hidden reasoning included: it limits the cost and the time.

Check your understanding

Predict: you send the T-80008 request with max_completion_tokens set to 250. What can the response look like, and how does your code know?

Show the answer

chat-small first spends output tokens on hidden reasoning; for this ticket it used 115 to 187 in the recorded runs. With 250, the answer can be complete, cut in the middle, or empty if the reasoning takes all 250. In a real test, all 250 tokens went to reasoning, and the content was empty. The code knows from the finish reason, “length”, and from the reasoning tokens in the usage, not from the status, which is 200.

Check your understanding

Tomás says: 'We tested temperature 0 with Gemma on Ollama, so we can use it with chat-small too.' What do you answer?

Show the answer

A setting that works on one model can fail on another. chat-small is a reasoning model and accepts only its default temperature: the real request with temperature 0 got HTTP 400. Check each model’s documentation and send one real request with the exact settings before you build on them. The same is true for the name of the output limit: Ollama ignored max_completion_tokens.

Recap

• A request has an endpoint, a credential (the API key in the Authorization header), a model identifier, messages with roles (system, user, assistant) and generation settings such as max_completion_tokens and temperature.
• A response has a status and headers, an ID, the model that answered, the answer as an assistant message, the finish reason and the usage.
• finish_reason: "length" means that the answer is cut or empty, even with status 200. For T-80008 with a limit of 40, all 40 tokens were hidden reasoning.
• Settings differ between models: chat-small rejects temperature: 0 with HTTP 400, and Ollama ignores max_completion_tokens. Test one real request first.
• python -m assistant.raw T-80008 prints the exact body. The SDK sends the same JSON and gives typed objects.

Next, you put all the provider code behind one boundary, keep the key out of the code, and make failures clear.

Next: The application boundary