Skip to content
Local LLM Course

How Models Are Made and Improved

Part 2 explained what a language model is. This part explains where one comes from, what each stage of its making cost, what can still be done to it afterwards, and who is allowed to do that.

It is deliberately a map rather than a manual. Nothing here is a recipe you run today; every technique named has a whole part of its own later on. What the map buys you is judgement. When a model card says “post-trained with GRPO on verifiable rewards”, when a forum post says a 4-bit quantisation “loses nothing”, when a release is called open source and its licence has a user cap in it, you want to know what is being claimed before you spend a week acting on it.

The five lessons follow a model’s life. Pretraining is where the knowledge comes from and where almost all of the money goes. Post-training is where a text predictor becomes something you can give an instruction to. The small-model toolkit is how the behaviour of a large model gets into one that fits on your desk, which is the whole premise of running models locally. Inference is the arithmetic that says how fast your machine will generate text, and it is the most useful piece of arithmetic in the course. Licences are the part everyone skips, and the part that decides whether you can ship what you built.

Then a reality check: you test the claim that a small local model is as good as the frontier, on your own machine, on twenty tasks you score blind, and you write down whichever answer you get.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • describe what a base model is trained on and roughly what that training cost, and say why a compute-optimal budget balances model size against training tokens;
  • name the post-training stages in order, say what each one changes, and read a model card’s description of its own post-training without needing a glossary;
  • explain distillation, pruning and quantisation, say which of the three removes parameters and which only rounds them, and predict which one a given problem needs;
  • work out a ceiling on decode speed for a given model on a given machine from memory bandwidth and bytes of active weights, and explain why prefill and decode behave differently;
  • read a model licence well enough to answer the two questions that matter, may I use this commercially and what must I put in my product, and say what “open weight” does not mean;
  • turn a vague claim about model quality into a test that could refute it, run it, and report the result you got rather than the one you expected.

What you need first

Section titled “What you need first”

Part 1’s vocabulary of loss, gradients, tensors, memory bandwidth and number formats, and Part 2’s account of tokens, context and how a transformer produces one token at a time. Nothing else. The five lessons need no hardware at all.

The reality check needs a machine with at least 12 GB of memory and about an hour, and it is the first page in the course where you download a model and generate text. It uses Ollama as a one-command way to do that, because the engines the course actually teaches come later: Part 6 covers llama.cpp and Part 7 covers Ollama and LM Studio properly.

How to work through it

Section titled “How to work through it”

Read the five lessons in order; each one uses the previous one’s vocabulary, and the licence lesson refers back to every model named in the other four. Each lesson ends with a pointer to the part of Level 3 or Level 4 that teaches the same subject with your hands on it, so this part doubles as an index into the second half of the course.

Do the reality check with a terminal open and your lab notebook from Part 1 to hand. It takes about forty-five minutes, most of which is the machine generating while you do something else, and it produces the first honest measurement you will have made about model quality.

Part3 of 27Level1 — AI LiteratePages6Estimated time3.1 hours

0 / 6 lessons in this part completed