Skip to content
Local LLM Course

The Training Toolchain

Level 2 ended with a working local AI service: models you can run, serve, benchmark and put to work. Level 3 is about changing the models themselves. Pretraining a small one from scratch, fine-tuning a large one on your own examples, aligning it with preferences, distilling it into something smaller, quantising it and measuring honestly what you did to it.

Every one of those techniques uses the same handful of libraries, the same memory arithmetic and the same discipline about writing down what you ran. This part teaches them once, on a run that finishes in minutes, so that the six parts after it can be about the technique rather than about the tooling. If you have ever followed a fine-tuning tutorial and spent the afternoon on wheel versions and out-of-memory errors instead of on the training, this part is the antidote.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • name the six libraries a training recipe is built from, say what each is responsible for, and know which one to read when something breaks;
  • install a working PyTorch training stack on your own platform track, verify that a real training step runs on the accelerator, and know what the honest support status of your track is and on what date it was checked;
  • compute, before downloading anything, how much memory a full fine-tune, a LoRA run and a QLoRA run of a given model need, and therefore which of the three your machine can do;
  • get text into the exact shape a trainer expects, apply a model’s chat template, decide which tokens the loss is computed on, and build a validation split that can still detect memorisation;
  • record a run so that its result still means something a week later, and reproduce it as closely as the hardware allows;
  • fine-tune a small model end to end, watch the loss fall, export the result to GGUF and run it in llama.cpp.

What you need first

Section titled “What you need first”

Level 2, in practice rather than in principle. You need the Python environment from Part 1 and the accelerator build of PyTorch for your track; a llama.cpp checkout built as in Part 6, for the export step in the lab; and the lab notebook you started in Part 1, because this part defines the run-log format that every later lab appends to it.

The memory arithmetic in this part extends the inference arithmetic from Part 4. If “weights are parameter count times bytes per parameter” and the key-value cache formula are not familiar, read that lesson again first: training arithmetic is the same idea with four more terms.

Nothing here needs a large machine. The lab’s memory floor is 8 GB, the model it fine-tunes has about six hundred million parameters, and the run finishes in minutes on every track. The larger models appear only in the arithmetic, where they cost nothing to reason about.

How to work through it

Section titled “How to work through it”

Read the first lesson for the map of the libraries, then do the environment lesson with a terminal open: it ends with a script that runs a real training step and tells you what it cost. The memory lesson is the one to come back to, and the one the later parts cite most; work through its arithmetic on your own machine’s memory rather than reading the tables passively. The dataset lesson is where most fine-tunes are quietly ruined, so read it before you build any data of your own. The tracking lesson defines a run-log format and ships a small helper the later labs import.

Then do the lab. It is deliberately small: a six-hundred-million-parameter model, about two hundred instruction examples, a few minutes of training and one exported GGUF file. The point is not the model you get. The point is that when Part 13 asks you to fine-tune something that matters, every step of the machinery will already be familiar.

Part11 of 27Level3 — Model BuilderPages6Estimated time3.5 hours

0 / 6 lessons in this part completed