Skip to content
Local LLM Course

Full Curriculum

Twenty-seven parts and a capstone, 184 pages, five difficulty levels, and about 126 hours of time on task including labs, projects and the capstone.

The five levels

Section titled “The five levels”

Levels are a dependency graph rather than parallel tracks. Level 3 assumes Level 2 because you cannot evaluate a fine-tune you cannot serve.

Level 1 — AI Literate

4 parts23 pages13 h

Explain how a language model works, what its numbers mean, what a licence allows, and which model fits which machine.

0 of 4 parts started

1. Machine Learning Foundations for LLM Work2. What a Language Model Is3. How Models Are Made and Improved4. The Open-Weight Model Landscape

Level 2 — Local Operator

6 parts41 pages26 h

Install, run, serve, benchmark and put to work open-weight models on your own hardware with the major engines.

0 of 6 parts started

5. Your Hardware: Four Platforms, One Course6. First Inference with llama.cpp7. Ollama, LM Studio and the Desktop Experience8. Platform-Native Engines: MLX, TensorRT-LLM and ROCm9. Serving at Scale: vLLM and SGLang10. Putting Models to Work

Level 3 — Model Builder

7 parts50 pages35 h

Pretrain a small model, fine-tune, align with preferences and reinforcement learning, distil, quantise and evaluate honestly.

0 of 7 parts started

11. The Training Toolchain12. Pretraining a Small Model from Scratch13. Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-Tunes14. Preference Optimisation and Reinforcement Learning15. Knowledge Distillation: Big Teacher, Small Student16. Quantisation and Evaluation, Properly17. Faster Inference: Speculative Decoding, Caching and Draft Models

Level 4 — Cluster Architect

6 parts33 pages23 h

Run models bigger than one machine, split prefill from decode, and operate the result as a service.

0 of 6 parts started

18. Beyond One Machine: Parallelism and Networks19. Clustering with llama.cpp RPC20. NVIDIA Clusters: DGX Spark Pairs, vLLM Multi-Node and Multi-GPU Desktops21. Apple Clusters: MLX Distributed and exo over Thunderbolt 522. Disaggregated Serving: Prefill Machines and Decode Machines23. Operating a Local AI Service

Level 5 — Agentic Engineer

5 parts37 pages30 h

Drive coding agents and build agent systems on local models, then improve the models with what the agents produce.

0 of 5 parts started

24. Tool Calling, MCP and the Agent Loop25. Coding Agents on Local Models26. Building Agent Systems27. Improving a Model for Your AgentsCapstone: Build and Defend Your Own Local AI Platform

Page types

Section titled “Page types”
Type What it is
Lesson Teaches something, with a quiz where there is anything worth testing
Lab You run something on your track and record the result
Project You build a complete, reusable piece of your platform
Challenge Something is broken; you diagnose it with evidence before reading the solution
Reality check A popular claim about local AI, made measurable and measured
Overview A part’s introduction and contents

Hands-on pages carry their platform tracks, their memory floor and, where the primary path needs two or more machines, a cluster badge. Every cluster page has a single-machine path.

The whole course

Section titled “The whole course”

Part 1Machine Learning Foundations for LLM WorkLevel 1 · AI Literate2.8 h

The handful of ideas from machine learning that every later decision rests on: loss, gradients, generalisation, tensors and precision.

Part 2What a Language Model IsLevel 1 · AI Literate3.4 h

Next-token prediction, tokens, embeddings, attention and the transformer, taught so that the reader can look at a checkpoint and know what every number is for.

Part 3How Models Are Made and ImprovedLevel 1 · AI Literate3.1 h

Pretraining, post-training, the small-model toolkit, inference physics, and the licences that decide what you may do with the result.

Part 4The Open-Weight Model LandscapeLevel 1 · AI Literate3.3 h

Who publishes what, how to read a model name, a model card and a benchmark, and how to pick a model that fits a memory budget.

Part 5Your Hardware: Four Platforms, One CourseLevel 2 · Local Operator4.5 h

The four machine classes the labs are written for, what makes each different, and how to get each one ready and measured.

Part 6First Inference with llama.cppLevel 2 · Local Operator4.3 h

The engine that runs on every one of the four platforms, its file format, its quantisation types, its server, and the flags that actually matter.

Part 7Ollama, LM Studio and the Desktop ExperienceLevel 2 · Local Operator3.5 h

The two tools most people start with, what they do on top of llama.cpp and MLX, the front-ends that sit on them, and a private chat service for your home network.

Part 8Platform-Native Engines: MLX, TensorRT-LLM and ROCmLevel 2 · Local Operator3.0 h

The engines built by or for each platform vendor, when they beat the portable ones, and the specialist engines for specific problems.

Part 9Serving at Scale: vLLM and SGLangLevel 2 · Local Operator5.2 h

The engines built for many users at once: batching, paged attention, prefix caching, server-side tool calling, speculative decoding, and a gateway project that the rest of the course builds on.

Part 10Putting Models to WorkLevel 2 · Local Operator5.2 h

Prompting local models properly, structured output, coding assistance in the editor, retrieval-augmented generation, multimodal inputs, safety beyond localhost, and the evaluation harness the rest of the course uses.

Part 11The Training ToolchainLevel 3 · Model Builder3.5 h

PyTorch, the Hugging Face libraries, a working training environment on each platform, the memory arithmetic of training, dataset handling, and reproducible experiments.

Part 12Pretraining a Small Model from ScratchLevel 3 · Model Builder4.7 h

The full pipeline from raw text to a model that can complete a sentence, run at a scale that fits one machine and one afternoon.

Part 13Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-TunesLevel 3 · Model Builder6.5 h

Teaching a model a format, a style or a domain with labelled examples, using adapters when memory is tight, and exporting the result to every engine.

Part 14Preference Optimisation and Reinforcement LearningLevel 3 · Model Builder5.5 h

From imitating examples to optimising for what you prefer, and then to reinforcement learning with rewards a program can verify.

Part 15Knowledge Distillation: Big Teacher, Small StudentLevel 3 · Model Builder6.8 h

Moving capability from a model that fits your 128 GB machine into one that fits your laptop, by logits, by sequences and by on-policy correction.

Part 16Quantisation and Evaluation, ProperlyLevel 3 · Model Builder5.1 h

The methods that shrink a model, the precision formats of current hardware, and the measurements that say what was lost.

Part 17Faster Inference: Speculative Decoding, Caching and Draft ModelsLevel 3 · Model Builder2.5 h

The techniques that make a served model faster without changing its answers, including training your own draft model.

Part 18Beyond One Machine: Parallelism and NetworksLevel 4 · Cluster Architect2.9 h

Why one box runs out, the five kinds of parallelism and what network each needs, home-cluster networking from 2.5 GbE to RDMA, and the reference cluster the rest of the level is built on.

Part 19Clustering with llama.cpp RPCLevel 4 · Cluster Architect3.9 h

The cross-platform way to run a model across machines: rpc-server on every box, one client, a model bigger than any single one of them, over TCP or RDMA.

Part 20NVIDIA Clusters: DGX Spark Pairs, vLLM Multi-Node and Multi-GPU DesktopsLevel 4 · Cluster Architect3.4 h

The NVIDIA path to models beyond one machine: two Sparks over ConnectX-7 with the official playbooks, vLLM across nodes with Ray, and what a multi-GPU desktop can and cannot do without NVLink.

Part 21Apple Clusters: MLX Distributed and exo over Thunderbolt 5Level 4 · Cluster Architect2.9 h

Two or more Macs as one model server: MLX's distributed backends, RDMA over Thunderbolt 5 on macOS 26.2 and later, exo's automatic partitioning, and a fair comparison with a GPU box.

Part 22Disaggregated Serving: Prefill Machines and Decode MachinesLevel 4 · Cluster Architect6.0 h

Separating the compute-heavy prompt phase from the bandwidth-heavy generation phase onto different machines, moving the KV cache between them, and measuring whether it helped.

Part 23Operating a Local AI ServiceLevel 4 · Cluster Architect4.1 h

Routing, observability, capacity and cost, security, backup and upgrades: running what you built as a service other people depend on.

Part 24Tool Calling, MCP and the Agent LoopLevel 5 · Agentic Engineer4.3 h

What an agent is, function calling end to end on local engines, the Model Context Protocol, reasoning models in loops, context engineering, and a minimal agent built by hand.

Part 25Coding Agents on Local ModelsLevel 5 · Agentic Engineer7.8 h

The agentic coding tools of 2026, which local models can drive them, how each is pointed at a local endpoint, sandboxing, and a workstation the reader keeps.

Part 26Building Agent SystemsLevel 5 · Agentic Engineer4.7 h

Frameworks, multi-agent patterns, agentic retrieval, evaluation, safety, and a system that uses a big model on the cluster and a small one for routing.

Part 27Improving a Model for Your AgentsLevel 5 · Agentic Engineer3.1 h

Closing the loop: the trajectories your agents produce become training data, the tests they run become rewards, and a big agent model becomes a small local one.

CapstoneCapstone: Build and Defend Your Own Local AI PlatformLevel 5 · Agentic Engineer10.3 h

A complete local AI platform on the reader's own hardware, with measured numbers, an improved small model, an agentic workstation, and a report that says what worked and what did not.