Skip to content
Local LLM Course

Tool Calling, MCP and the Agent Loop

Level 5 is about making a local model do work rather than answer questions. Every tool in the rest of this level, from the coding agents in Part 25 to the multi-agent systems in Part 26, is the same small program: send the conversation and a list of functions to a model, read a structured request back, run the function, append the result, and go round again.

That program is about a hundred lines. It is worth writing yourself once, because everything that goes wrong in an agent goes wrong inside it, and the failures are quiet. A tool call that arrives as prose looks like a chatty model. A loop that never terminates looks like a slow model. A context window that fills up looks like a model that forgot. None of those raises an error, and none of them is diagnosable from outside the loop.

So this part builds the loop by hand, against the gateway from Part 9, on whichever machine you have. Then it connects the same tools over the Model Context Protocol, which is the standard that lets one tool server be used by your own agent and by a desktop application at the same time, and looks squarely at what that standard exposes you to.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • define an agent precisely, distinguish it from a workflow, and say which of the two a given problem actually needs;
  • send a tool schema to a local engine, get a parsed call back on vLLM, llama-server, Ollama or LM Studio, and diagnose the failure when you do not;
  • measure how reliably a given model and quantisation emit valid calls, and record the result in a form Part 25 can reuse;
  • explain what an MCP server, client and host are, which transport to use, and how to evaluate a third-party server before you let it near your files;
  • decide when a reasoning model earns its thinking tokens inside a loop and when it does not;
  • compute the key-value cache cost of an agent’s growing transcript, choose a context length for your track, and apply compaction, retrieval and prefix caching to keep it affordable;
  • write a working agent with three sandboxed tools, guard rails and a transcript log, and expose one of its tools over MCP to your own agent and to a desktop client.

What you need first

Section titled “What you need first”

Part 9’s gateway, or any OpenAI-compatible endpoint you can reach. The agent scripts here talk to a base URL and a model name and care about nothing else, so llama-server from Part 6, Ollama or LM Studio from Part 7, and vLLM from Part 9 all work.

Part 9’s lesson on server-side tool calling is the prerequisite for the second lesson here: it explains the chat template and the per-family parser, which this part assumes and does not repeat. Part 10’s document index is used as the agent’s search tool, and its lesson on privacy and security introduces the prompt injection this part turns into a working attack. Part 4’s memory arithmetic and Part 17’s prefix caching are both used directly in the context-engineering lesson.

Both labs run on all four tracks with 8 GB of memory, because the agent is a Python program and the model is already being served.

How to work through it

Section titled “How to work through it”

Read the five lessons in order. The second one is the longest and the most load-bearing: a model that emits invalid calls three times in ten makes every later part harder, and the reliability test in that lesson is the thing that tells you whether you have one.

Then do both labs with a terminal open and a server running. The first builds the agent; the second gives it a protocol. Keep both, because Part 25 compares your loop against real coding agents, Part 26 puts several of them together, and Part 27 fine-tunes a model to be better at exactly the calls your reliability test says it gets wrong.

Part24 of 27Level5 — Agentic EngineerPages7Estimated time4.3 hours

0 / 7 lessons in this part completed