Skip to content
Local LLM Course

The Open-Weight Model Landscape

Choosing a model is the decision a reader of this course will make most often, and it is the one most often made from a forum post. This part replaces the forum post with two things: a dated map of who publishes what, and the arithmetic that answers “will it fit and how much context can I afford” before anything is downloaded.

The map half is deliberately perishable. Every family description on the first lesson says when it was checked, because between the writing of that page and your reading of it several of those families will have shipped a new generation. That is not a flaw to apologise for; it is the subject. So the lesson teaches the refresh procedure alongside the map: which page to open, which fields to read, and which claims to distrust until you have measured them.

The arithmetic half does not perish. Bytes per parameter, the key-value cache formula and the headroom rules behave the same way in 2026 as they will in 2030, and every later part uses them. Part 6 uses them to pick a quantisation, Part 9 to size a serving deployment, Part 13 to work out whether a fine-tune fits, and Parts 18 to 22 to decide what has to be split across machines.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • name the main open-weight publishers, say what each family is used for in this course, and carry out the procedure that tells you whether the map has gone stale;
  • explain what a mixture-of-experts model is, distinguish total from active parameters, say why a hybrid attention model’s memory grows differently with context, and predict what each of these does to a memory budget on a unified-memory machine;
  • read a model name end to end and say what base, instruct, thinking, coder, vision, embedding, distilled and quantisation-aware each mean for the job you have;
  • read a model card for the fields that matter, name the benchmarks the cards quote, explain contamination, and restate any score you meet as a claim with an owner and a date;
  • compute the weights and key-value cache bytes for any model at any context length, decide what fits on your machine with sensible headroom, and download and organise the models that do.

What you need first

Section titled “What you need first”

Part 1’s lesson on tensors, GPUs and precision supplies bytes per parameter and the difference between capacity and bandwidth, which the memory lesson here builds on directly. Part 2 supplies layers, attention heads and parameter counts. Part 3 supplies prefill and decode, and the licence vocabulary that the families lesson leans on.

No model is downloaded until the lab, and the lab runs on the 8 GB tier. It states download sizes per memory tier so that a reader on a slow connection can choose a small shortlist and still complete every later part.

How to work through it

Section titled “How to work through it”

Read the five lessons in order. The architecture lesson and the memory lesson are a pair: the first explains why two models with the same parameter count can need very different amounts of memory, and the second turns that into arithmetic you can do on paper. The card-and-benchmark lesson is the one to reread whenever a new model release lands with a chart attached.

Then do the lab. It ends with a shortlist in your lab notebook and a model library directory that every engine in Parts 6 to 9 will share, so getting the layout right once here saves re-downloading tens of gigabytes later.

Part4 of 27Level1 — AI LiteratePages6Estimated time3.3 hours

0 / 6 lessons in this part completed