What an Agent Is: The Loop, Tools and State

By the end of this lesson you will be able to say what an agent is in terms of a program you could write, name the four things it is made of, tell an agent apart from a workflow and say which one a given problem needs, place any tool you are about to hand a model on a scale from harmless to irreversible, and predict where an 8B model running on your own machine will struggle in a loop.

The word is used for everything from a chatbot with a search box to a system that opens pull requests unattended. That vagueness is not harmless: it is why people are surprised both by what agents can do and by what they do wrong. So this lesson defines the thing narrowly.

The loop

An agent is a loop in which a model decides the next step.

That is the whole definition. Everything else is detail. The loop has four moving parts, and you can point at each one in the code you write in this part’s first lab.

One turn of an agent loop

1. 1The program sends state to the modelThe system prompt, the tool schemas, the task, and every observation so far. This is one ordinary chat-completions request; the model has no memory of the previous turn beyond what you resend.
2. 2The model emits either a tool call or an answerA structured request naming a function and its arguments, or ordinary text. Which one you get is the decision, and it is the only decision the model makes.
3. 3The program validates the callIs that a tool we offer? Do the arguments match its schema? Is this call allowed right now? A call that fails validation is an observation like any other, not a crash.
4. 4The program runs the tool and appends the resultThe result goes back as a tool message. This is the only way anything from outside the model reaches it.
5. 5The program checks its stopping conditionsTurn count, token count, wall clock, a repeated call, or the model calling the tool that means "done". If none fires, go round again.

Three of the five steps are the program's, not the model's. An agent is mostly control flow you wrote, wrapped around one decision you did not.

Notice what the model does not do. It does not run anything. It does not remember anything. It does not keep going on its own. Every one of those is your loop. When people describe an agent as “the model taking actions”, they have collapsed the diagram into its second step, and the collapse is where the surprises come from.

Tools are typed functions with documentation for a reader who is not a person

A tool is a function you expose to the model, described by a name, a natural-language description, and a JSON Schema for its arguments. The MCP specification’s tool definition is exactly that list: name, an optional title, a description (“Human-readable description of functionality”), an inputSchema that “MUST be a valid JSON Schema object”, and an optional outputSchema. The OpenAI-shaped tools array in a chat-completions request carries the same three things under type: "function".

Two properties of that description matter more than they look.

The description is the interface. The model chooses between your tools by reading their descriptions, and it fills in arguments by reading the schema. Anthropic’s essay on building agents makes the point that the effort you put into this deserves to be comparable to what you would put into a human-facing interface, and recommends “poka-yoke” designs that make the wrong call hard to express. A parameter called path with no constraint invites a path you did not want; a parameter documented as “a path relative to the workspace root, without ..” and validated on arrival does not.

The description is also an instruction the model will read. Whatever is in the description is inside the prompt. That is the mechanism the third lesson’s tool-poisoning section is about, and it is why the MCP specification says clients “MUST consider tool annotations to be untrusted unless they come from trusted servers”.

✦Tip

Fewer tools, more sharply separated, beat many overlapping ones. A model choosing between read_file and search_documents is making a decision the descriptions can make obvious. A model choosing between search, find, lookup and query is guessing, and its accuracy will show up in the reliability test in the next lesson as a wrong-tool rate rather than a malformed-call rate.

Where state lives

The model is stateless. Every request carries everything it is going to know. So “state” in an agent means: what does the program put into the next request, and where does it keep the rest?

Four places, in increasing distance from the model:

The transcript. The list of messages you resend every turn: system prompt, task, each assistant message, each tool result. This is the agent’s working memory, it grows with every step, and it is the thing that runs out. The fifth lesson is about that.

The scratchpad. A file the agent writes to and reads back with its own tools. Anything written here leaves the context window and comes back only when asked for, which is the point.

Retrieval. An index the agent searches, exactly as in Part 10. Facts live outside the context and arrive when a query matches.

The environment. The filesystem, the database, the repository. The agent changes this, and unlike the other three it does not reset when the loop ends. This is where consequences live.

✕Common mistake

Treating the transcript as memory. A long-running agent that keeps everything in the transcript will hit the context limit, and the usual response is to raise --ctx-size, which raises the key-value cache cost per token and slows every turn. The transcript is the wrong place for anything the agent will not need on the very next turn. Moving observations to a scratchpad file and re-reading them by name is cheaper, and the fifth lesson does the arithmetic.

Agent or workflow

Anthropic’s essay draws a line worth adopting, because it makes the design decision explicit. Workflows are “systems where LLMs and tools are orchestrated through predefined code paths”. Agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks”.

The essay names five workflow patterns: prompt chaining, routing, parallelisation, orchestrator-workers and evaluator-optimiser. Every one of them has the same property, which is that you can draw the sequence before the program runs. It reserves agents for “open-ended problems where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path”.

That distinction is a cost decision, not a taste one.

	Workflow	Agent
Number of model calls	Known before it runs	Unknown; bounded only by your limits
Cost and latency	Predictable	Varies by an order of magnitude between runs
Failure mode	A step fails, visibly	It wanders, plausibly
Debugging	Read the code path	Read the transcript
Right when	The steps are the same every time	The steps depend on what is found

A classification job with a fixed schema is a workflow, and building it as an agent buys you nothing but variance. “Find why this test is failing” is an agent, because the second step depends on what the first one printed.

?Reality check

The claim that agents supersede workflows is worth holding up. Both essays this lesson cites say the opposite: the simplest structure that solves the problem is the one to use, and an agent is what you reach for when the sequence genuinely cannot be written down. Most production systems described as agents are workflows with one agentic step inside them, and that is usually the right shape.

Autonomy is a dial, and it has a ratchet

“Autonomous” is not a property of the loop. It is a property of what you let the loop do without asking, and it is worth writing down as a ladder because each rung is a different risk.

Levels of autonomy, from safest to least reversible

each rung removes a human from the path

1. SuggestThe model proposes a call; nothing runs. Useful for measuring a model before you trust it.no consequences
2. Confirm each callEvery call is shown with its arguments and waits for a keypress. Slow, and the right default for anything new.human in the loop
3. Allow-list without confirmationRead-only or reversible tools run unattended; everything else confirms. This is where most useful local agents sit.
4. Unattended within a sandboxAnything goes, inside a container or a dedicated user with a scratch directory and no credentials. The blast radius is the sandbox.you defined the blast radius
5. Unattended on the real systemThe agent writes to your files, your repository, your network. Nothing in a model makes this safe; only the environment can.not taught in this course

The MCP specification puts the same idea in normative language: for tools, there SHOULD always be a human in the loop with the ability to deny tool invocations, and hosts SHOULD present confirmation prompts.

The ratchet is that people move down this ladder when an agent is working well and never move back up when it starts failing differently. A confirmation prompt you have pressed enter on two hundred times has stopped being a control. Design the permission boundary so that it is enforced by what the tool can reach, not by your attention: an allow-listed command set, a directory the file tool cannot escape, a network the sandbox does not have.

⛨Security

Two rungs of that ladder are the same call with a different consequence. run_command with an allow-list of git status, pytest and ls is a read-only tool. The same function without the allow-list is a shell. OWASP’s prompt-injection guidance lists “enforce privilege control and least privilege access” and “require human approval for high-risk actions” among its mitigations, and both are properties of the tool, not of the prompt. The prompt is the part an attacker gets to write.

Where a local model fits, and where it struggles

An agent asks a model for something quite specific: pick one of these functions, fill in its arguments correctly, and stop when the job is done. That is a narrower skill than conversation, and it degrades differently.

What local models in this course’s reference set do well:

• Single tool calls with small argument sets. Qwen3-8B and Qwen3-4B both emit Hermes-style calls their engines parse, and a well-described tool with two string parameters is usually filled correctly.
• Short loops. Three to eight turns, with observations that fit in a few hundred tokens each, is well within what an 8B model keeps track of.
• Structured, repetitive work. Extract, classify, look up, report. The kind of thing the fourth lesson argues does not need a reasoning model at all.

Where they struggle, in roughly the order you will meet it:

• Knowing when to stop. The most common failure of a small model in a loop is not a wrong call, it is calling the same tool again with the same arguments because nothing told it the task was finished. Your loop must have a limit, and your tools should include an explicit way to declare completion.
• Parallel calls. vLLM’s own documentation is blunt about the variation here: parallel tool calls “are not supported for Llama 3, but it is supported in Llama 4 models”, Mistral 7B “struggles to generate parallel tool calls correctly”, and Llama’s smaller models “frequently fail to emit tool calls in the correct format”. Assume sequential unless you have measured otherwise.
• Long transcripts. As observations accumulate, the instruction at the top of a long context competes with a great deal of text, and adherence falls off before anything looks broken.
• Resisting instructions inside tool results. A small model given a document that says “ignore your previous instructions” is not reliably going to decline. This is not a quantisation problem or a size problem you can fix; it is the reason the second lab tests for it and the reason tools are sandboxed rather than trusted.

✱Hardware note

The loop itself costs nothing. The agent scripts in this part are plain Python talking to an endpoint over HTTP, so every track runs them identically and the memory floor is set by the model you were already serving. What differs between tracks is how long a turn takes and how much context you can afford, and both of those are the subject of the fifth lesson.

Define a state machine with bounded effects

An agent loop can be described as states: obtain model output, validate the proposed action, authorise it, execute it, record the observation and decide whether to continue. A final answer, invalid call, exhausted budget, cancellation or tool failure each needs an explicit transition.

Write a maximum for turns, elapsed time, tokens and repeated identical calls. These limits address different failure modes. A loop can spend few tokens but wait indefinitely on a tool, or make many cheap repeated calls without progressing. Store enough state to explain why it stopped.

For a first design, use read-only tools and a task with an independently checkable answer. Then add one reversible action and test denied arguments, malformed calls and a tool exception. The model proposes the next step; the application owns permissions and execution. A useful agent is a system whose loop reaches a verifiable outcome under defined constraints. A transcript that looks busy or ends with “done” is not itself evidence that the task succeeded.

Recap

An agent is a loop in which a model decides the next step, and three of the five steps in that loop are code you wrote: assembling the request, validating the call, and deciding whether to stop. Tools are typed functions whose descriptions and schemas are the model’s entire interface to them, and whose descriptions are also text the model will follow. State lives in four places with very different costs: the transcript, which grows and is expensive; a scratchpad; a retrieval index; and the environment, which is the only one that does not reset. A workflow is a fixed sequence you can draw in advance and an agent is not, and choosing the agent when a workflow would do buys variance rather than capability. Autonomy is a ladder from suggestion to unattended action, enforced properly by what a tool can reach rather than by a confirmation prompt you have stopped reading. And a local model in a loop is good at short sequences of well-described single calls, and unreliable at stopping, at parallel calls, at long transcripts and at refusing instructions that arrive inside a tool result.

Check your understanding

Question 1. In one turn of an agent loop, which of these does the model itself do?

• Runs the tool and reads its output
• Decides whether to emit a tool call or an answer, and with what arguments
• Decides whether the loop should stop
• Remembers the previous turn without being told

Show the answer and why

Answer: Decides whether to emit a tool call or an answer, and with what arguments

The model makes exactly one decision per turn and emits tokens describing it. Assembling the request, validating and running the call, appending the result and checking the stopping conditions are all the program. The model has no memory of previous turns except the transcript you resend.

Question 2. You need to extract five fields from each of two thousand invoices, in the same way every time. Agent or workflow?

• An agent, because it involves a model and tools
• A workflow: the sequence is known before it runs, so a fixed code path is cheaper, faster and easier to debug than a loop that decides its own steps
• An agent, because two thousand documents is a lot
• Neither; this needs a fine-tuned model first

Show the answer and why

Answer: A workflow: the sequence is known before it runs, so a fixed code path is cheaper, faster and easier to debug than a loop that decides its own steps

Anthropic's distinction is about whether the path can be predicted. Here it can: read, extract against a schema, validate, write. Building it as an agent adds unpredictable cost and a wandering failure mode without adding any capability. Agents earn their keep when the next step depends on what the last one found.

Question 3. Which of these are properties of the tool rather than of the prompt, and therefore actually enforceable? Select all that apply.

• An allow-list of commands the run tool will execute
• A system prompt asking the model not to delete files
• A file tool that refuses any path outside one directory
• A per-call timeout and memory limit on the sandbox

Show the answer and why

Answer: An allow-list of commands the run tool will execute, A file tool that refuses any path outside one directory, A per-call timeout and memory limit on the sandbox

A system prompt is a request to a token predictor and an attacker gets to write text that competes with it. An allow-list, a path check and a resource limit are code that runs regardless of what the model was persuaded to ask for. Design the boundary where it can be enforced.

Question 4. A small local model in a loop keeps calling the same search tool with the same query, turn after turn. What is the most likely cause?

• The model is broken and needs re-downloading
• Nothing in the loop tells it the task is complete or stops it, and repeating a call is a common failure of small models with no explicit completion path
• The tool schema is invalid
• The context window is too large

Show the answer and why

Answer: Nothing in the loop tells it the task is complete or stops it, and repeating a call is a common failure of small models with no explicit completion path

Knowing when to stop is the weakest part of a small model in a loop. The fix is in the program: a maximum turn count, a check for a repeated identical call, and an explicit tool the model can call to declare it is finished. All three are in this part's first lab.

Check my answersClear