Local Large Language Models
- 28
- Parts
- 184
- Pages
- 5
- Levels
- 126
- Hours
- 1,896
- Citations
0 / 184 lessons completed
Progress tracking needs browser storage, which is unavailable here. The course works exactly the same without it.
What this course is
Section titled “What this course is”A complete course in local large language models: what they are, how they are made, how to run the open-weight ones on your own machines, how to make a small one better, how to run one that is bigger than any single machine you own, and how to put them to work as coding agents and agent systems.
It is taught on four kinds of hardware, side by side: the NVIDIA DGX Spark, the AMD Ryzen AI Max+ 395 mini PCs such as the GMKtec EVO-X2, Apple silicon Macs, and ordinary desktops and laptops with NVIDIA GPUs. Every lab carries instructions for all four, and every cluster lab carries a path for a reader with one computer.
What makes it different from most material on the subject is the thing it refuses to do.
Who it is for
Section titled “Who it is for”- Engineers and systems administrators who run infrastructure and want to run models the same way: understood, measured and operated.
- Developers who want a coding agent that runs on their own machine, and want to know which models can actually drive one.
- People with a capable machine who have run Ollama or LM Studio and want to know what is happening underneath, and what else the machine could do.
- Anyone who wants to train, fine-tune, distil or align a model with their own hands and see what changes.
Prerequisites
Section titled “Prerequisites”- Comfort with a command line on Linux, macOS or Windows. The course is terminal-first.
- Some Python. Enough to read a script and change a value; the training parts teach the rest.
- Ordinary numeracy. No machine-learning or statistics background is assumed; Part 1 builds what you need.
- A machine from one of the four tracks. The what you need page says what each tier can follow.
What you will be able to do
Section titled “What you will be able to do”By the end you will be able to:
- Explain how a language model works well enough to reason about memory, speed and cost.
- Compute whether a model fits a machine at a given quantisation and context, and predict its decode speed before downloading it.
- Install, serve and benchmark open-weight models with llama.cpp, Ollama, LM Studio, MLX, vLLM and SGLang, and choose between them with evidence.
- Pretrain a small model from raw text, fine-tune one with adapters, align it with preferences and reinforcement learning, distil a large local teacher into a small student, quantise the result and measure what was lost.
- Run a model larger than any one of your machines across two or three of them, and split prompt processing from generation across machines that suit each.
- Drive coding agents such as Aider, Codex CLI, OpenCode and Claude Code with a local model, build your own agents, sandbox them, and improve the model with the transcripts they produce.
- Report all of it in a form that someone else could check.
The curriculum
Section titled “The curriculum”27 parts plus a capstone, 184 pages, five levels, from AI Literate to Agentic Engineer. Each level is a set of things you can do at the end of it, and the levels are a dependency graph rather than parallel tracks.
Level 1 — AI Literate
Explain how a language model works, what its numbers mean, what a licence allows, and which model fits which machine.
Level 2 — Local Operator
Install, run, serve, benchmark and put to work open-weight models on your own hardware with the major engines.
Level 3 — Model Builder
Pretrain a small model, fine-tune, align with preferences and reinforcement learning, distil, quantise and evaluate honestly.
Level 4 — Cluster Architect
Run models bigger than one machine, split prefill from decode, and operate the result as a service.
Level 5 — Agentic Engineer
Drive coding agents and build agent systems on local models, then improve the models with what the agents produce.
The whole thing is on the curriculum page, with time estimates derived from the pages rather than from marketing.
Hardware and platform tracks
Section titled “Hardware and platform tracks”Required: one machine from one of the four tracks, with at least 8 GB of GPU or unified memory, and enough disk for the models a tier can run.
Optional: a second machine, of any track, for the cluster parts at their intended scale; the 128 GB machines for the 30B-class teachers and the 120B-class models; Thunderbolt 5 Macs or a DGX Spark pair for the vendor-supported clusters.
About 126 hours of time on task across the whole course, including labs, projects and the capstone, derived by summing the per-page estimates in the curriculum manifest. Unattended time, while a model trains or a dataset generates, is stated separately on each lab.
At an hour a day that is roughly four months. Most people will not do all of it, and the course is built so that the parts are useful separately.
Progress tracking
Section titled “Progress tracking”Your progress and your chosen platform track are stored in this browser only, using local storage. There are no accounts, no sign-in and nothing is sent anywhere. Clear your browser data and they are gone; open the site on another device and it starts fresh. If storage is unavailable, everything else works exactly the same.
Disclaimer
Section titled “Disclaimer”- New here? How to use this course explains the structure, the callouts, the code labels, the platform tracks and how to pick a starting point.
- Want the map? The full curriculum.
- Ready to set up? What you need.
- Want to know how it was made? About this course.