Skip to content
Local LLM Course

Supervised Fine-Tuning: LoRA, QLoRA and Full Fine-Tunes

This is the part most readers came for. You have a model that is nearly right: it answers well but will not hold your output format, or it writes in a register that is wrong for your documents, or it handles your domain’s vocabulary as though it had never seen it. Supervised fine-tuning is the technique that closes gaps of that kind, and on the hardware this course is written for it is entirely practical.

It is also the technique with the widest gap between what people expect and what it does. A fine-tune reliably changes how a model responds. It is a poor way to teach a model new facts, and it can quietly damage abilities you were not measuring. So the part is built around one honest question, asked before and after every run: did it get better on my evaluation set? The set you built in Part 10 is the instrument, and it is the reason this part can make claims at all.

The second half of the part is the step that tutorials skip. A training run produces an adapter directory, which no engine in Level 2 can serve. Merging it, converting it, quantising it, checking that the converted model still behaves like the one you trained, and putting it behind the gateway from Part 9 is what turns an experiment into something you use on Tuesday morning.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • say what supervised fine-tuning reliably changes and what it does not, and choose between fine-tuning, retrieval and a better prompt for a given problem, with a reason;
  • explain low-rank adaptation well enough to set rank, alpha, dropout and target modules deliberately, and say what QLoRA quantises and what it does not;
  • build a supervised fine-tuning dataset in a format a trainer accepts, deduplicate it, decontaminate it against your evaluation set, and hold out a split you never train on;
  • run the reference recipe with TRL and PEFT, read the training and evaluation loss curves, stop at the right checkpoint, and recognise overfitting from the logs rather than from a hunch;
  • choose between Unsloth, Axolotl, LLaMA-Factory, NeMo AutoModel and mlx-lm for your platform and your task, and say which of them the documentation supports on your machine, on a stated date;
  • merge an adapter, export to GGUF or MLX or an AWQ or FP8 checkpoint, prove the export matches the model you trained, and serve it through your own gateway;
  • fine-tune a small model to follow your own output format, measure the change against the base model on your Part 10 task set, and defend the result;
  • diagnose a fine-tune that scored worse than the model it started from, and prove the fix.

What you need first

Section titled “What you need first”

Part 11 in full. This part uses its libraries, its memory arithmetic, its dataset shapes and its run-log format, and it refers to its reference recipe by name rather than repeating it. If you have not installed a working training stack for your track and watched a real training step run, do that first: the failures are much easier to read when the environment is not one of the suspects.

Part 10’s evaluation harness, in the form of your own task file. Every claim in this part is measured with run-eval.py and judge.py against tasks you wrote. If you skipped that lab, the lab, the project and the challenge here all have nothing to measure against.

The gateway from Part 9 is useful rather than required. It is where the fine-tuned model ends up, and having one stable name in front of every engine is what makes the before-and-after comparison a matter of changing a string rather than restarting servers.

The models are small on purpose. The lab fine-tunes a model between one and four billion parameters with a memory floor of 12 GB, and the project between four and eight billion with a floor of 16 GB, using a quantised base where memory is tight. Nothing here needs a machine larger than the smallest in the course.

How to work through it

Section titled “How to work through it”

Read the first three lessons before you train anything. The first decides whether fine-tuning is the right tool for your problem at all, and readers who skip it spend an afternoon teaching a model facts it will not retain. The second is the mechanism, and its arithmetic is what tells you which of the three methods your machine can run. The third is where most fine-tunes are quietly ruined, because a dataset that overlaps your evaluation set produces a number that looks like success and is not.

The fourth and fifth lessons are the toolchain: the reference recipe in detail, then the higher-level tools that wrap it and which platforms each one documents. The sixth is the export path, and it is the one to come back to whenever an adapter has to become a model.

Then do the lab with a terminal open. It is the smallest complete fine-tune the course can build: your own format, a few hundred examples, a run that finishes inside an hour on every track, and a number that says whether it worked. The project repeats it on a domain you choose and asks you to defend the result in writing. The challenge hands you four faults, one at a time, and a procedure for finding them.

Part13 of 27Level3 — Model BuilderPages9Estimated time6.5 hours

0 / 9 lessons in this part completed