Reasoning Models in Agent Loops

A reasoning model writes a private working-out before it answers. In a single-turn chat that costs you one wait. In an agent loop it costs you one wait per turn, and a loop that takes eight turns has paid for eight of them before anything useful happened.

By the end of this lesson you will be able to switch thinking on and off for every engine and model family in this course, decide which turns deserve a thinking budget, say what to do with reasoning text when you build the next request, and make the case for and against using a non-reasoning model as your agent.

What each family offers, and how to switch it

Four different mechanisms, and no two spell it the same way.

Qwen3 ships both modes in one model. The card describes “seamless switching between thinking mode (for complex logical reasoning, math, and coding) and non-thinking mode (for efficient, general-purpose dialogue)”, with thinking wrapped in <think>...</think> blocks and on by default. It is turned off with enable_thinking=False at the template level, and there are soft switches: /think and /no_think in the prompt, with the model following the most recent instruction in a multi-turn conversation. The card also gives different sampling settings per mode, which most people miss: temperature 0.6, top-p 0.95, top-k 20 and min-p 0 for thinking, with an explicit “DO NOT use greedy decoding”, and temperature 0.7, top-p 0.8, top-k 20 and min-p 0 for non-thinking.

gpt-oss has a dial rather than a switch. The card describes three levels of reasoning effort and states that “the reasoning level can be set in the system prompts, e.g., ‘Reasoning: high’”. Its thinking lives on the harmony format’s analysis channel, with function calls placed on the commentary channel and the answer on final.

Qwen3-Coder does not think at all: the card says it “supports only non-thinking mode and does not generate <think></think> blocks in its output”. That is a deliberate choice for a model built for agentic coding, and it is the strongest hint in this lesson.

Everything else is per-model. Read the card before you assume, because the failure when you get it wrong is silent: a model that was trained to think and is prevented from doing so answers worse, and a model that was not trained to think and is asked to will produce something that looks like thinking and is not.

Per engine, the switches as documented on 2026-09-09:

Engine	Turn thinking off	Where the thinking arrives
vLLM	chat_template_kwargs: {"enable_thinking": false} per request, or --default-chat-template-kwargs '{"enable_thinking": false}' at launch	A reasoning field, when --reasoning-parser is set
llama-server	--reasoning-budget 0, documented as “immediate end”	Controlled by --reasoning-format: unparsed in message.content, or in message.reasoning_content
Ollama	think: false in the request	message.thinking

⌚Version note

The vLLM reasoning page read on 2026-09-09 records a rename: “Reasoning used to be called reasoning_content. To migrate, directly replace reasoning_content with reasoning.” A client written for an older vLLM 0.28.0 · verified 2026-09-08 reads an empty field rather than getting an error. llama-server’s --reasoning-format and --reasoning-budget are read from the README on the same date and are not yet in this course’s captured command reference for llama.cpp v0.4.0 · verified 2026-09-08, so they appear in prose here rather than in a runnable block.

The cost is per turn, and it compounds

Think about where the time in a turn goes. The prefill reads the transcript, which grows every turn. The decode writes the thinking, then the call. The tool runs, which is usually fast. Then it all happens again with a longer transcript.

Where a turn goes, with and without thinking, over four turns of the same task

No thinking

prefillcalltoolprefillcalltoolprefillcalltoolprefillanswer

Thinking every turn

prefillthinkcalltoolprefillthinkcalltoolprefillthinkcalltoolprefillthinkanswer

Thinking on turn one only

prefillplancalltoolprefillcalltoolprefillcalltoolprefillanswer

Relative shapes, not measurements: the blocks are proportional to token counts, and decode is where the wall-clock goes on every track in this course. The prefill blocks grow because the transcript grows. Thinking on the planning turn buys most of the benefit for a quarter of the cost.

Two numbers set the scale, and both come from the model cards rather than from a stopwatch. Qwen3’s card recommends a maximum output length of 32,768 tokens, rising to 38,912 for complex maths and programming, and thinking is what fills that budget. A tool call is a few dozen tokens. So a thinking turn can decode two or three orders of magnitude more tokens than the call it eventually emits, and decode is bandwidth-bound on every track in this course.

Pending validationTurn cost with and without thinking, Qwen3-8B on the eight-turn agent task set

Configuration	Output tokens per turn	Turns to finish	Wall clock, whole task (s)
Thinking off	pending	pending	pending
Thinking on, every turn	pending	pending	pending
Thinking on first turn only	pending	pending	pending

per track, recorded by the validation pass, per track · llama.cpp v0.4.0 · Qwen3-8B, Q4_K_M · 32,768 tokens of context · pending

The first lab's task set and transcript log produce all three rows: run agent-tasks.json with thinking off, on, and on for the first turn only, and read the totals out of the transcript. Fill this in on your own hardware before deciding.

▤Memory budget

Thinking also costs memory, because every thinking token that stays in the transcript is a token of key-value cache on every subsequent turn. If you keep reasoning in the history, an eight-turn task with a few thousand thinking tokens per turn adds tens of thousands of tokens of context by the end. The next lesson does that arithmetic for each reference model; the short version is that thinking you keep is charged for again on every turn that follows it.

Where a thinking budget is worth spending

Not every turn is the same kind of turn. In practice an agent loop has three:

Planning turns, where the model decides what the task actually requires and in what order. There is real work here, the input is ambiguous, and a wrong decision costs several wasted turns. This is where thinking pays.

Dispatch turns, where the next step is obvious from the last observation: the file was found, now read it. There is nothing to reason about. Thinking here produces a paragraph explaining that reading the file is a good idea, and then reads the file.

Recovery turns, where something failed: a tool returned an error, a search found nothing, a test broke. The input is genuinely surprising and the right response is not obvious. Thinking often pays here too.

So the useful pattern is not “thinking on” or “thinking off” but thinking sometimes, decided by your loop:

Fragment — not complete on its own

# Spend thinking where a decision is actually being made, not on every dispatch.
def wants_thinking(turn_index, last_observation):
    if turn_index == 0:
        return True                       # the planning turn
    if last_observation is None:
        return False
    return "error" in last_observation.lower() or "no results" in last_observation.lower()

payload["chat_template_kwargs"] = {"enable_thinking": wants_thinking(turn, last)}

That is a policy you can measure. Run the first lab’s task set three ways, thinking always, never and selectively, and compare turns to completion and wall clock. The answer differs by model and by task, which is the whole point of measuring it.

✦Tip

When you do switch modes per request, switch the sampling settings with them. The Qwen3 card gives different recommendations for the two modes and warns against greedy decoding in thinking mode. A loop that sets temperature once at startup and then toggles enable_thinking is running one of the two modes outside its documented settings.

Interleaved thinking, and what the engines actually do with it

“Interleaved thinking” describes a loop where the model thinks, calls a tool, sees the result, thinks again about that result, and calls another. It is the natural shape for a recovery turn, and it is worth being precise about what happens to those thoughts because the engines differ.

On every engine in this course, each turn is an independent request. There is no thinking that persists between turns unless you put it there. So interleaving is not a mode you switch on; it is what you get by default when thinking is enabled and the loop runs more than once. What you choose is whether turn three can see what turn two was thinking, and that choice is one line in your loop.

Two things make the difference worth measuring rather than assuming. Keeping the thinking gives later turns the reasoning that led to the current state, which sometimes stops a model re-deciding something it already settled. Dropping it keeps the transcript short, keeps the prefix cache intact, and removes a large block of text that competes for attention with the observations. On the short, well-tooled tasks in this part’s first lab, dropping it is almost always the better trade; on a fifteen-turn task with an ambiguous goal, it is worth trying both and comparing turns to completion.

What to do with the reasoning text

Three choices, and the middle one is usually right.

Drop it. Read the reasoning field for your log, and build the next request from the assistant message and the tool results only. The transcript stays short, the cache stays warm, and no thinking token is ever charged twice. Most local agent loops should do this.

Keep a summary. Have the loop write one line into the transcript: what the model decided and why. This is a form of the compaction the next lesson covers, and it keeps continuity across turns without keeping thousands of tokens.

Keep everything. Only when the model’s own documentation says its multi-turn behaviour depends on seeing its previous thinking. Check the card; do not assume in either direction.

There is a fourth thing you should not do, which is feed reasoning back as if it were content. vLLM states that “tool calling only parses functions from the content field, not from the reasoning”, so reasoning text pasted into a content field is text the parser will read as an answer rather than a call.

⛨Security

Reasoning text is not private. It is generated by the same model from the same context, it can repeat anything the model was given, and if you log transcripts it lands in your log file. If a document with personal data went into the context, expect it to appear in the working-out. Part 10’s retention advice applies to reasoning fields exactly as it applies to prompts: decide what you keep and for how long, before you start keeping it.

When a non-reasoning model is the better agent

The case is stronger than it sounds, and Qwen3-Coder is the evidence: a model built specifically for agentic coding, with a 262,144-token native context, that does not think at all.

Prefer a non-reasoning model when the loop has these properties, which most tool loops do:

• The next step is determined by the last observation. Dispatch turns dominate. There is nothing for reasoning to add.
• Turn count matters more than single-turn quality. Ten cheap turns often beat three expensive ones, because each turn brings back real information from a tool rather than speculation.
• The tools are the intelligence. A search index, a test runner and a type checker each know something the model does not. Reasoning about what the tests might say is strictly worse than running them.
• The transcript is long. Anthropic’s context-engineering post describes an “attention budget” that thins as context grows, and thinking tokens spend that budget on text that is about to be discarded.

Prefer a reasoning model when the task genuinely needs a plan before any tool is useful, when the observations are ambiguous and need interpretation, or when a wrong first move is expensive to undo.

?Reality check

The claim that a reasoning model is simply better for agents is worth testing rather than assuming. The measurement is not benchmark quality, it is turns to completion and wall clock on your own tasks, which the first lab’s transcript log gives you for free. On short, well-tooled tasks a non-thinking model that takes six quick turns frequently finishes before a thinking model has completed its third. On genuinely open-ended tasks the order reverses. Both results are common enough that only your own task set settles it.

Spend reasoning effort where the outcome can improve

An agent turn may require planning a difficult change, or merely reading a file whose path is already known. Using a long generation budget on every turn can increase latency and consume context without improving those routine steps. Compare a fixed task suite under deliberately chosen reasoning configurations and keep tool behaviour unchanged.

Record final success, tool errors, generated tokens and wall time. Inspect cases where extra generation changes the plan and cases where it only lengthens the transcript. A model that writes a convincing analysis but invokes the wrong tool has still failed the application contract.

Keep any provider-specific reasoning representation separate from the user-visible answer and tool messages according to the API contract. Do not assume a displayed explanation is a faithful causal account of internal computation. For correctness, rely on independent checks of actions and outputs. The useful setting is the one that improves verified outcomes within the task’s resource budget, with stopping conditions that prevent repeated deliberation from becoming an unbounded loop.

Recap

Qwen3 carries both modes with enable_thinking and the /think and /no_think switches and different recommended sampling settings for each, gpt-oss takes a reasoning-effort level in the system prompt and puts thinking on a harmony channel, and Qwen3-Coder does not think at all. Each engine spells the switch differently: chat_template_kwargs on vLLM, --reasoning-budget 0 on llama-server, think: false on Ollama, and the field the thinking comes back in has recently been renamed on vLLM. In a loop the cost of thinking is paid once per turn and again on every following turn if you keep it in the transcript, so the right question is which turns deserve it: planning and recovery turns usually do, dispatch turns usually do not, and a loop can decide that per request. Drop the reasoning text from the next request or keep one line of summary, never paste it into a content field, and remember it can contain anything the context contained. And a non-reasoning model is often the better agent, because in a well-tooled loop the tools carry the knowledge and the transcript is the scarce resource.

Check your understanding

Question 1. An eight-turn agent task runs with thinking on and the whole reasoning text kept in the transcript. What happens to the cost of turn eight compared with turn one?

• It is the same; each turn is independent
• The prefill grows with every kept thinking block, so turn eight re-reads all the earlier reasoning before it can start, and its key-value cache is much larger
• It falls, because the model has already decided what to do
• Only the decode is affected

Show the answer and why

Answer: The prefill grows with every kept thinking block, so turn eight re-reads all the earlier reasoning before it can start, and its key-value cache is much larger

Everything in the transcript is resent every turn. Kept reasoning is charged again on every subsequent turn, in prefill time and in cache memory. Dropping it, or replacing it with a one-line summary, is the cheapest single change available to a thinking agent.

Question 2. Which of these are documented ways to turn thinking off for the engines this course uses? Select all that apply.

• chat_template_kwargs with enable_thinking false on vLLM
• --reasoning-budget 0 on llama-server, documented as immediate end
• think: false in an Ollama request
• Setting temperature to 0

Show the answer and why

Answer: chat_template_kwargs with enable_thinking false on vLLM, --reasoning-budget 0 on llama-server, documented as immediate end, think: false in an Ollama request

The first three are documented switches. Temperature has nothing to do with it, and for Qwen3 the card explicitly warns against greedy decoding in thinking mode, so turning temperature down is a change in the wrong direction as well as the wrong lever.

Question 3. Your agent runs a search tool, gets three results, and needs to read the most relevant one. What kind of turn is this and what should the loop do?

• A planning turn: spend a large thinking budget
• A dispatch turn: the next step follows from the observation, so thinking adds latency without adding a decision
• A recovery turn: retry the search
• It does not matter; thinking is always worth it

Show the answer and why

Answer: A dispatch turn: the next step follows from the observation, so thinking adds latency without adding a decision

Dispatch turns dominate most loops and are exactly where thinking is wasted. Reserve the budget for the first turn, where the plan is made, and for turns after something went wrong, where the right response is genuinely unclear. A loop can decide this per request.

Question 4. Why is Qwen3-Coder relevant to the argument that a non-reasoning model can be the better agent?

• It is smaller than the reasoning models
• It is a model built for agentic coding whose card states it supports only non-thinking mode and does not generate think blocks, alongside a very long native context
• It cannot use tools
• It has a different licence

Show the answer and why

Answer: It is a model built for agentic coding whose card states it supports only non-thinking mode and does not generate think blocks, alongside a very long native context

A model designed for exactly this workload was shipped without a thinking mode and with a 262,144-token native context. That is a design decision worth noticing: in a loop the tools carry the knowledge and the context is the scarce resource, so tokens spent on private working-out compete with tokens spent on observations.

Check my answersClear