Your Progress

Mark a page complete using the control at the bottom of it, and it appears here.

!Important

Everything on this page is stored in this browser only.

There is no account, no sign-in and nothing is sent anywhere. Progress, and your chosen platform track, live in your browser’s local storage, which means:

• Clearing your browser data erases them.
• Opening the site on another device or another browser starts fresh.
• In private browsing they may not persist beyond the session.
• If site data is blocked entirely, the page says so and everything else works exactly the same.

There is no export and no sync, deliberately. Adding one would mean adding an account, and an account would mean collecting something about you that this course has no reason to hold.

Where you are

1

Level 1 — AI Literate

4 parts · 23 pages · 13 h

Part 1: Machine Learning Foundations for LLM WorkPart 2: What a Language Model IsPart 3: How Models Are Made and ImprovedPart 4: The Open-Weight Model Landscape

0 of 4 parts started

1. Machine Learning Foundations for LLM Work2. What a Language Model Is3. How Models Are Made and Improved4. The Open-Weight Model Landscape

2

Level 2 — Local Operator

6 parts · 41 pages · 26 h

Part 5: Your Hardware: Four Platforms, One CoursePart 6: First Inference with llama.cppPart 7: Ollama, LM Studio and the Desktop ExperiencePart 8: Platform-Native Engines: MLX, TensorRT-LLM and ROCmPart 9: Serving at Scale: vLLM and SGLangPart 10: Putting Models to Work

0 of 6 parts started

5. Your Hardware: Four Platforms, One Course6. First Inference with llama.cpp7. Ollama, LM Studio and the Desktop Experience8. Platform-Native Engines: MLX, TensorRT-LLM and ROCm9. Serving at Scale: vLLM and SGLang10. Putting Models to Work

3

Level 3 — Model Builder

7 parts · 50 pages · 35 h

Part 11: The Training ToolchainPart 12: Pretraining a Small Model from ScratchPart 13: Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-TunesPart 14: Preference Optimisation and Reinforcement LearningPart 15: Knowledge Distillation: Big Teacher, Small StudentPart 16: Quantisation and Evaluation, ProperlyPart 17: Faster Inference: Speculative Decoding, Caching and Draft Models

0 of 7 parts started

11. The Training Toolchain12. Pretraining a Small Model from Scratch13. Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-Tunes14. Preference Optimisation and Reinforcement Learning15. Knowledge Distillation: Big Teacher, Small Student16. Quantisation and Evaluation, Properly17. Faster Inference: Speculative Decoding, Caching and Draft Models

4

Level 4 — Cluster Architect

6 parts · 33 pages · 23 h

Part 18: Beyond One Machine: Parallelism and NetworksPart 19: Clustering with llama.cpp RPCPart 20: NVIDIA Clusters: DGX Spark Pairs, vLLM Multi-Node and Multi-GPU DesktopsPart 21: Apple Clusters: MLX Distributed and exo over Thunderbolt 5Part 22: Disaggregated Serving: Prefill Machines and Decode MachinesPart 23: Operating a Local AI Service

0 of 6 parts started

18. Beyond One Machine: Parallelism and Networks19. Clustering with llama.cpp RPC20. NVIDIA Clusters: DGX Spark Pairs, vLLM Multi-Node and Multi-GPU Desktops21. Apple Clusters: MLX Distributed and exo over Thunderbolt 522. Disaggregated Serving: Prefill Machines and Decode Machines23. Operating a Local AI Service

5

Level 5 — Agentic Engineer

5 parts · 37 pages · 30 h

Part 24: Tool Calling, MCP and the Agent LoopPart 25: Coding Agents on Local ModelsPart 26: Building Agent SystemsPart 27: Improving a Model for Your AgentsCapstone: Capstone: Build and Defend Your Own Local AI Platform

0 of 5 parts started

24. Tool Calling, MCP and the Agent Loop25. Coding Agents on Local Models26. Building Agent Systems27. Improving a Model for Your AgentsCapstone: Build and Defend Your Own Local AI Platform

Every page

0 / 184 lessons completed

Reset all progress

Part 1Machine Learning Foundations for LLM Work

0 / 5 lessons in this part completed

• ○What Learning Means: Data, Loss and Gradient Descent
• ○Neural Networks, Activations and Backpropagation
• ○Generalisation: Train, Validation, Test and Overfitting
• ○Tensors, GPUs and Precision: Why Matrix Multiplication Is Everything
• ○Lab: Your Python Environment and a First Trained Model

Part 2What a Language Model Is

0 / 6 lessons in this part completed

• ○From Autocomplete to Assistant: Next-Token Prediction
• ○Tokens, Tokenisers and Vocabulary
• ○Embeddings: Meaning as Geometry
• ○Attention and the Transformer
• ○Parameters, Layers and Model Size
• ○Lab: Look Inside a Model

Part 3How Models Are Made and Improved

0 / 6 lessons in this part completed

• ○Pretraining: Learning from Trillions of Tokens
• ○Post-Training: SFT, Preference Tuning and Reinforcement Learning
• ○Distillation, Pruning and Quantisation: How Small Models Get Good
• ○Inference: Prefill, Decode and Why Memory Bandwidth Rules
• ○Open Weights, Open Source and Licences
• ○Reality Check: 'A Small Local Model Is as Good as the Frontier'

Part 4The Open-Weight Model Landscape

0 / 6 lessons in this part completed

• ○Model Families and Who Makes Them
• ○Dense, Mixture-of-Experts and Hybrid Architectures
• ○Base, Instruct, Thinking, Coder, Vision, Embedding: Reading a Model Name
• ○Reading a Model Card and a Benchmark
• ○Choosing a Model for a Memory Budget
• ○Lab: Build Your Model Shortlist

Part 5Your Hardware: Four Platforms, One Course

0 / 7 lessons in this part completed

• ○Why Memory, Not FLOPS, Decides What You Can Run
• ○NVIDIA DGX Spark: GB10, DGX OS and the aarch64 Caveat
• ○AMD Ryzen AI Max+ 395: Strix Halo, ROCm and Vulkan
• ○Apple Silicon: Unified Memory, Metal and MLX
• ○NVIDIA Desktops and Laptops: VRAM Tiers, CUDA and WSL2
• ○Lab: Prepare Your Machine
• ○Lab: Measure Your Memory Bandwidth and Compute

Part 6First Inference with llama.cpp

0 / 7 lessons in this part completed

• ○llama.cpp: The Engine That Runs Everywhere
• ○Installing and Building llama.cpp on Your Platform
• ○GGUF and Quantisation Types
• ○llama-cli and llama-server
• ○Sampling: Temperature, Top-p, Min-p, Repetition and Determinism
• ○Lab: Run and Benchmark the Course Reference Models
• ○Challenge: The Model That Runs at Two Tokens per Second

Part 7Ollama, LM Studio and the Desktop Experience

0 / 6 lessons in this part completed

• ○Ollama: Models as a Service
• ○LM Studio: GUI, MLX and Headless Serving
• ○Front-Ends: Open WebUI and Friends
• ○Managing a Model Library: Storage, Naming and Versions
• ○Lab: A Private Chat Service for Your Home Network
• ○Reality Check: 'The Default Context Is Enough'

Part 8Platform-Native Engines: MLX, TensorRT-LLM and ROCm

0 / 5 lessons in this part completed

• ○MLX and mlx-lm: Apple's Native Path
• ○TensorRT-LLM, NIM and the DGX Spark Playbooks
• ○AMD-Native: ROCm Builds, Lemonade Server and the NPU Question
• ○Specialist Engines: ExLlamaV3, ktransformers and mistral.rs
• ○Lab: Same Model, Every Engine

Part 9Serving at Scale: vLLM and SGLang

0 / 8 lessons in this part completed

• ○Why a Second Kind of Engine: Batching, Paged Attention and Throughput
• ○Installing vLLM: x86 CUDA, DGX Spark, ROCm and What Does Not Work
• ○Serving with vLLM: Quantised Weights, Context, Memory and Multi-GPU
• ○SGLang: When to Choose It
• ○Tool Calling and Structured Output on the Server Side
• ○Speculative Decoding: Draft Models, EAGLE and n-gram
• ○Lab: Serve a Model to Twenty Concurrent Users
• ○Project: Your Local Model Gateway

Part 10Putting Models to Work

0 / 8 lessons in this part completed

• ○Prompting That Works Locally: System Prompts, Chat Templates and Thinking Modes
• ○Structured Output and JSON Mode
• ○Local Coding Assistants: Autocomplete and Chat in Your Editor
• ○Retrieval-Augmented Generation: Embeddings, Chunking and Vector Stores
• ○Vision, Speech and Documents: Multimodal Locally
• ○Privacy, Security and Serving Beyond localhost
• ○Project: A Private Document Question-Answering Service
• ○Lab: Benchmark Local Models on Your Own Tasks

Part 11The Training Toolchain

0 / 6 lessons in this part completed

• ○PyTorch, Transformers, Datasets and the Hugging Face Ecosystem
• ○Setting Up a Training Environment on Each Platform
• ○Memory Arithmetic for Training: Weights, Gradients, Optimiser States and Activations
• ○Datasets: Formats, Chat Templates, Tokenisation and Packing
• ○Experiment Tracking and Reproducibility
• ○Lab: Your First Training Run

Part 12Pretraining a Small Model from Scratch

0 / 6 lessons in this part completed

• ○What Pretraining Teaches You That Fine-Tuning Cannot
• ○Anatomy of a Training Loop: nanochat Line by Line
• ○Data for Pretraining: FineWeb-Edu, Cosmopedia and Training a Tokeniser
• ○Scaling Laws and Compute Budgets at Home
• ○Lab: Train a 10M to 125M Parameter Model in an Afternoon
• ○Project: A Domain Micro-Model

Part 13Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-Tunes

0 / 9 lessons in this part completed

• ○What Fine-Tuning Changes and What It Cannot
• ○LoRA and QLoRA Explained
• ○Building an SFT Dataset
• ○Fine-Tuning with TRL and PEFT
• ○Unsloth, Axolotl, LLaMA-Factory and mlx-lm: Higher-Level Tools
• ○Merging, Exporting and Quantising a Fine-Tuned Model
• ○Lab: Fine-Tune a 1B to 4B Model to Follow Your Format
• ○Project: A Specialist Assistant
• ○Challenge: The Fine-Tune That Got Worse

Part 14Preference Optimisation and Reinforcement Learning

0 / 8 lessons in this part completed

• ○From Imitation to Preferences: Why Ranking Beats Copying
• ○DPO and Its Family: IPO, KTO, ORPO and SimPO
• ○Lab: DPO a Model to Prefer Your Style
• ○Reinforcement Learning with Verifiable Rewards: GRPO Explained
• ○Reward Functions: Maths, Code Tests, Format and Length
• ○The RL Toolchain: TRL, Unsloth, verl, OpenRLHF and vLLM Rollouts
• ○Lab: GRPO on a Maths or Code Task on One Machine
• ○Reality Check: 'RL Makes Small Models Reason'

Part 15Knowledge Distillation: Big Teacher, Small Student

0 / 9 lessons in this part completed

• ○Why Distillation Works
• ○Three Kinds of Distillation: Logit, Sequence and On-Policy
• ○Reasoning Distillation: Teacher Traces as Training Data
• ○Generating Synthetic Data with a Local Teacher
• ○Lab: Sequence-Level Distillation of a 30B-Class Teacher into a 4B Student
• ○Lab: Logit Distillation with TRL's Distillation Trainers
• ○Pruning and Compression: Prune-and-Distil
• ○Challenge: The Student That Learned the Teacher's Mistakes
• ○Project: The Distillation Pipeline

Part 16Quantisation and Evaluation, Properly

0 / 8 lessons in this part completed

• ○Post-Training Quantisation in Depth: GPTQ, AWQ, K-quants, imatrix and HQQ
• ○Quantisation-Aware Training and the Blackwell Formats: FP8, NVFP4 and MXFP4
• ○Measuring Quantisation Damage: Perplexity, KL Divergence and Task Evaluations
• ○Evaluation Harnesses: lm-evaluation-harness, lighteval, EvalPlus and Your Own
• ○LLM-as-Judge, Contamination and Honest Reporting
• ○Lab: Quantise Your Fine-Tune Five Ways and Measure Each
• ○Lab: Run a Standard Benchmark Suite on Your Model
• ○Challenge: The Benchmark That Lied

Part 17Faster Inference: Speculative Decoding, Caching and Draft Models

0 / 4 lessons in this part completed

• ○Speculative Decoding Revisited: Acceptance Rates and When It Pays
• ○Prefix Caching and KV Reuse
• ○Training a Draft Model: Medusa and EAGLE at Home
• ○Lab: Train and Deploy a Draft for Your Model

Part 18Beyond One Machine: Parallelism and Networks

0 / 5 lessons in this part completed

• ○Why One Box Runs Out: The Memory Wall and the Bandwidth Wall
• ○Tensor, Pipeline, Expert, Data and Sequence Parallelism
• ○Networking for Home Clusters: 2.5 GbE to 200 GbE, Thunderbolt 5 and RDMA
• ○The Course Reference Cluster and the Single-Machine Path
• ○Lab: Build and Measure Your Cluster Network

Part 19Clustering with llama.cpp RPC

0 / 5 lessons in this part completed

• ○llama.cpp RPC: Layers Across Machines
• ○RDMA Transport, Tuning and Measuring the Split
• ○Lab: Run a Model Bigger Than Any One Machine
• ○Lab: A Mixed-Platform Cluster
• ○Challenge: The Cluster That Is Slower Than One Machine

Part 20NVIDIA Clusters: DGX Spark Pairs, vLLM Multi-Node and Multi-GPU Desktops

0 / 5 lessons in this part completed

• ○Connecting Two DGX Sparks over ConnectX-7
• ○vLLM Multi-Node with Ray: Tensor Parallel Inside, Pipeline Parallel Across
• ○Multi-GPU Desktops: PCIe, Tensor Parallel Without NVLink and Expert Parallel
• ○TensorRT-LLM and Dynamo on Spark Pairs
• ○Lab: Serve a 400B-Class Model on Two DGX Sparks

Part 21Apple Clusters: MLX Distributed and exo over Thunderbolt 5

0 / 4 lessons in this part completed

• ○mlx.distributed: Ring, MPI and RDMA over Thunderbolt 5
• ○exo: Automatic Partitioning Across Your Macs
• ○Lab: A Two-Mac Cluster over Thunderbolt 5
• ○Reality Check: 'Four Mac Studios Replace a GPU Server'

Part 22Disaggregated Serving: Prefill Machines and Decode Machines

0 / 7 lessons in this part completed

• ○Why Prefill and Decode Want Different Hardware
• ○The KV Cache as a Transferable Object: Mooncake, LMCache and NIXL
• ○vLLM Disaggregated Prefill: Connectors and the Proxy
• ○SGLang PD Disaggregation and NVIDIA Dynamo
• ○Lab: Two-Machine Prefill and Decode with vLLM
• ○Lab: KV Cache Offload and Sharing
• ○Project: A Tiered Inference Architecture

Part 23Operating a Local AI Service

0 / 7 lessons in this part completed

• ○Routing and Model Management: llama-swap, LiteLLM, NGINX and Health Checks
• ○Observability: Metrics, Logs and Traces for LLM Serving
• ○Capacity Planning and Cost per Million Tokens at Home
• ○Security for Exposed Endpoints and the Model Supply Chain
• ○Backup, Upgrades and Reproducibility of a Model Estate
• ○Lab: Dashboards for Your Cluster
• ○Challenge: The 3 a.m. Out-of-Memory

Part 24Tool Calling, MCP and the Agent Loop

0 / 7 lessons in this part completed

• ○What an Agent Is: The Loop, Tools and State
• ○Function Calling End to End on Local Engines
• ○Model Context Protocol: Servers, Clients and Transports
• ○Reasoning Models in Agent Loops
• ○Context Engineering: Memory, Compaction and the KV Budget
• ○Lab: A Minimal Agent from Scratch
• ○Lab: Write and Connect an MCP Server

Part 25Coding Agents on Local Models

0 / 11 lessons in this part completed

• ○The Landscape: Terminal, Editor and Autonomous Agents
• ○Which Local Models Can Actually Drive an Agent
• ○Aider with Local Models
• ○OpenAI Codex CLI and OpenCode with Local Models
• ○Claude Code with a Local Endpoint
• ○Editor Agents: Cline, Kilo Code, Continue and the Proprietary Editors
• ○Goose and OpenHands: Autonomous Agents and Sandboxing
• ○Lab: One Task, Six Agents
• ○Lab: Sandbox Your Agent: Containers, Permissions and Secrets
• ○Challenge: The Agent That Escaped the Sandbox
• ○Project: A Local Agentic Coding Workstation

Part 26Building Agent Systems

0 / 7 lessons in this part completed

• ○Agent Frameworks Compared
• ○Multi-Agent Patterns: Router, Planner and Executor, Critic, Parallel Fan-Out
• ○Agentic Retrieval and Research Agents
• ○Evaluating Agents: Trajectories, Success Rates and Cost
• ○Safety: Prompt Injection, Tool Permissions and Human-in-the-Loop
• ○Project: A Multi-Agent System on Your Cluster
• ○Reality Check: 'Agents Are Just Loops'

Part 27Improving a Model for Your Agents

0 / 5 lessons in this part completed

• ○Collecting Trajectories from Your Agents
• ○Fine-Tuning for Tool Use and Your Codebase
• ○Reinforcement Learning on Agent Tasks: Tests as Rewards
• ○Distilling a Big Agent Model into a Small Local One
• ○Lab: Fine-Tune a Small Model on Your Own Agent Trajectories and Measure the Gain

CapstoneCapstone: Build and Defend Your Own Local AI Platform

0 / 7 lessons in this part completed

• ○Capstone Overview and Assessment Criteria
• ○Capstone 1: Hardware and Model Plan
• ○Capstone 2: The Inference Service
• ○Capstone 3: Cluster or Tiered Deployment
• ○Capstone 4: Improve a Small Model
• ○Capstone 5: The Agentic Workstation
• ○Capstone 6: The Report