Lab: Fine-Tune a 1B to 4B Model to Follow Your Format

Validated on: written from the documentation cited above; not yet validated on hardware on any track. The base model, tool versions, wall-clock and per-track notes each track was executed with belong here once the validation pass has run this lab on real machines.

Objective

By the end of this lab you will have a fine-tuned model that answers in an output format you chose, a number saying how much better it holds that format than the model it started from, a second number saying whether anything else got worse, an exported file your engines can serve, and a name for it in your gateway.

The deliverable is not the model. It is the pair of numbers and the run record behind them. A fine-tune with no measurement is a story; a fine-tune with a before-and-after on tasks you wrote is a result.

⌂Single-machine path

Single-machine on every track. One model trains at a time, and the evaluation talks to whatever is serving over HTTP, which may be the same box.

Requirements

Seventy-five minutes, of which about fifty-five are attended. The unattended parts are the model download, the training run itself and the GGUF conversion. The memory floor is 12 GB, which is the 1.7-billion-parameter base with a rank-16 adapter; the 4-billion-parameter base wants the tier above.

You need the training environment from Part 11 for your track, a llama.cpp checkout built as in Part 6 for the export step, your own task file from Part 10, and about 25 GB of free disk for the base model, the merged model and two GGUF files.

The base models are Apache-2.0 licensed; the model reference records the licence and the Qwen3-4B card states it as apache-2.0. Pick by memory tier:

Your memory	Base model	Adapter	Why
12 GB	Qwen3-1.7B	rank 16, all seven projections	The floor. Trains in minutes and the format change is still clearly visible.
16 to 24 GB	Qwen3-4B	rank 16	Enough capacity that the fine-tune holds the format on inputs unlike the training ones.
32 GB and above	Qwen3-4B	rank 32, longer sequences or a larger batch	Spend the room on sequence length before you spend it on rank.

The floor: LoRA on Qwen3-1.7B, rank 16, batch 1 at 1,024 tokens, on a 12 GB machine

Frozen base weights, BF16

3.4 GB

Adapter weights, gradients and Adam states

0.3 GB

Activations and logits

0.4 GB

Reserved for the operating system

2 GB

Free

5.9 GB

Total

12 GB

Estimate from Part 11's arithmetic, not a measurement. The frozen base is the BF16 size from the model reference; the adapter carries 16 bytes per trainable parameter; activations use the checkpointed formula plus the logits tensor; the reserve is an allowance for the operating system. There is room here for a larger batch, which is the first thing to try if the run is slow.

Track S — NVIDIA DGX Spark

The primary path: TRL and PEFT on CUDA, with transformers 5.16.1 · verified 2026-09-08, TRL 1.12.0 · verified 2026-09-08 and PEFT 0.20.0 · verified 2026-09-08 from Part 11’s environment lesson. With 128 GB of unified memory you can hold the base model, the training run and a serving process at once, which makes the evaluation step in task 7 a matter of starting a second server rather than stopping the first.

Unsloth is a documented option on this machine and its guide is worth reading, but note that the guide’s Docker image pins older versions of transformers and trl than this course does. If you try it, treat it as a second run with its own record rather than as a drop-in swap.

Track X — AMD Ryzen AI Max+ 395Partial

TRL and PEFT on ROCm PyTorch is the primary path here. Unsloth's AMD page (read 2026-09-09) names RDNA 3/3.5/4 RX 6000-9000 cards and the MI300X and does not name this chip, so it is worth trying and recording rather than expecting.

Use TRL and PEFT on the ROCm PyTorch build from Part 11. PyTorch reports a ROCm device as cuda, so train-lora.py needs no changes and its device line will say cuda.

Two things are documented for this chip and worth knowing. bitsandbytes’ installation page lists gfx1151 among its ROCm wheel targets from ROCm 6.4.4 onwards, so a 4-bit path exists here even though this lab does not need one. And the GPU-visible memory cap from Part 5 applies to training as it does to inference: the machine’s total is not the budget, the GPU’s share is.

If you try Unsloth, its AMD page documents uv pip install unsloth[amd] with ROCm 6.0 or newer. Record what happens with the date, because that is a data point the documentation does not have.

Track M — Apple silicon

Use mlx-lm, not PyTorch. train-lora-mlx.sh wraps mlx_lm.lora, and the dataset generator writes the mlx-lm layout alongside the TRL one, so the two paths train on the same examples.

Two flags matter here. --num-layers is the memory knob, documented with a default of 16; lowering it adapts fewer layers for less memory and less capacity. --mask-prompt puts the loss on the completion only, which is what completion_only_loss does on the other tracks, and leaving it off trains the model to generate your questions as well as your answers.

Export is mlx_lm.fuse rather than a PEFT merge, and the script runs it for you. bitsandbytes has no GPU path on macOS, as its installation page shows Apple silicon only in the CPU table, so the QLoRA variant in this part’s project uses a different route on this track.

Track N — NVIDIA desktop or laptop

The primary path: TRL and PEFT on CUDA. On a 12 GB card use Qwen3-1.7B, batch 1 and gradient accumulation 8; on 16 GB and above, Qwen3-4B with the same settings; on 24 GB and above, raise the batch size before you raise the rank.

Inside WSL2, remember Part 6’s warning about the virtual machine’s memory limit: the training process sees the WSL2 allocation and not the Windows total, and an out-of-memory error that contradicts your arithmetic is usually that.

The lab, end to end

1. 1Decide the formatOne contract, written down, that every training example and every evaluation task will honour.
2. 2Generate the datasetA few hundred examples plus a held-out task file, in both the TRL and mlx-lm layouts.
3. 3DecontaminateAgainst your own Part 10 task file, so the regression check is measuring something.
4. 4TrainLoRA on your track, evaluating each epoch, stopping when the evaluation loss stops improving.
5. 5Read the curvesTwo numbers and their shape. The best epoch tells you whether the dataset and the epoch count agree.
6. 6Score against the baseSame tasks, same settings, same quantisation. The format set for the gain, your own set for the regression.
7. 7Export and serveMerge, convert, quantise, and give it a name in the gateway next to the base model.

Tasks 1 to 3 decide whether the result will mean anything, and take about half the attended time. Everything after task 4 is mechanical.

Tasks

Working directory and terminal roles

Prepare the course execution workspace once before this procedure. It includes this part's scripts, data and shared Python helpers. In the client or training terminal, select this directory:

RunnableAll tracks

select this part’s execution directory

export LABS_ROOT="${LABS_ROOT:-$HOME/llm-course/labs}"
export LAB_DIR="$LABS_ROOT/part-13-supervised-fine-tuning"
cd "$LAB_DIR"
pwd
test -f "make-format-dataset.py"

Expected result: pwd ends in part-13-supervised-fine-tuning and the file check returns successfully. If it does not, finish workspace preparation before continuing. Activate the environment in the requirements for your track. Bare script and data filenames below are relative to this directory; paths to earlier experiments must point at the artefacts you actually retained.

Keep each foreground server in a separate terminal and send requests from this terminal. Reapply lesson-specific environment variables in each new shell. Stop at the first failed checkpoint and retain its output; the execution guide explains how to distinguish missing files, endpoint failures and capacity problems.

1. Decide the format before you generate anything

Ten minutes with a text editor and nothing running.

Write down the output contract you want. Not “better summaries”: the shape, in enough detail that you could mechanically check it. The supplied template uses a triage format of exactly three lines, Summary:, Severity: and Action:, with a word limit and a list of strings that must never appear. Yours should be something you actually want, because you are going to serve this model afterwards.

Three properties make a format worth fine-tuning for. It has to be checkable without a model, so that the deterministic checks carry most of the score. It has to be something the base model gets wrong some of the time, or there is nothing to measure. And it has to be the same in every example, because the model learns the distribution you show it.

▷Exercise

Before you go on, ask the base model for the format ten times with a careful prompt and count how many times it complies. That number is your baseline in the crudest possible form, and if it is ten out of ten you should pick a harder format. This takes three minutes and it saves the hour you would otherwise spend fine-tuning for something prompting already solved.

2. Generate the dataset

RunnableAll tracks

make-format-dataset.py

"""Generate a format-following supervised fine-tuning dataset, and a held-out task file to score it.

Purpose: build a few hundred examples that all answer in one fixed output format, split them
         into a training set, a validation set and a held-out task file in the Part 10
         harness's shape, and write them in both the TRL and the mlx-lm layouts so either
         training path can read them. Every answer is derived from the structured fields
         that produced the question, so the labels are correct by construction rather than
         by an author's memory.
Platform: all (standard library only; the optional teacher step needs a served model)
Minimum memory: 8 GB on the machine running the optional teacher; this script needs very little
Assumes: Python 3.10 or newer. With --teacher-url, an OpenAI-compatible endpoint is reachable:
         llama-server from Part 6, the gateway from Part 9, or anything else speaking that API.
         The teacher is used only to rewrite the question text into more natural prose; the
         answers are never generated, because a label you did not check is not a label.

Usage: python3 make-format-dataset.py --out-dir . --count 320 --seed 0
       python3 make-format-dataset.py --print-template > my-format.json
       python3 make-format-dataset.py --template my-format.json --out-dir .
       python3 make-format-dataset.py --out-dir . \
           --teacher-url http://127.0.0.1:8080/v1 --teacher-model local/chat --teacher-count 80

Writes, under --out-dir:
  data/train.jsonl, data/valid.jsonl          TRL conversational prompt-completion
  data-mlx/{train,valid,test}.jsonl           mlx-lm completions
  format-tasks.json                           held-out tasks in the Part 10 task-file shape
  format-dataset.json                         everything generated, with its provenance
"""
from __future__ import annotations

import argparse
import json
import random
import sys
import urllib.error
import urllib.request
from pathlib import Path
from typing import Any

# ---------------------------------------------------------------------------
# The built-in template. Everything about the format lives here, so replacing the
# format means replacing this object rather than editing the generator: run with
# --print-template, edit the copy, and pass it back with --template.
# ---------------------------------------------------------------------------
TEMPLATE: dict[str, Any] = {
    "name": "ticket-triage",
    "system": (
        "You triage incoming reports for a small operations team. "
        "Answer only in the required format, with no preamble and no closing remark."
    ),
    "instruction": (
        "Triage this report. Reply with exactly three lines and nothing else:\n"
        "Summary: one line, under twelve words\n"
        "Severity: one of low, medium or high\n"
        "Action: one imperative sentence"
    ),
    "answer_format": "Summary: {summary}\nSeverity: {severity}\nAction: {action}",
    "must_contain": ["Summary:", "Severity:", "Action:"],
    "must_not_contain": ["```", "Sure,", "Here is", "Certainly"],
    "max_words": 40,
    "reporters": [
        "A customer", "The overnight operator", "A colleague in support",
        "The monitoring system", "A developer on the platform team", "The duty manager",
    ],
    "systems": [
        {"name": "the billing API", "short": "billing"},
        {"name": "the document index", "short": "the index"},
        {"name": "the model gateway", "short": "the gateway"},
        {"name": "the nightly backup job", "short": "the backup job"},
        {"name": "the internal wiki", "short": "the wiki"},
        {"name": "the report generator", "short": "the report generator"},
    ],
    "symptoms": [
        {
            "text": "is returning errors for roughly one request in ten",
            "summary": "{short} failing intermittently",
            "severity": "high",
            "action": "Page the on-call engineer and check the error rate for {short}.",
        },
        {
            "text": "is unreachable from every machine we have tried",
            "summary": "{short} completely unreachable",
            "severity": "high",
            "action": "Page the on-call engineer and confirm whether {short} is running.",
        },
        {
            "text": "answers correctly but takes several times longer than usual",
            "summary": "{short} responding slowly",
            "severity": "medium",
            "action": "Check load and recent changes on {short} before escalating.",
        },
        {
            "text": "logged one failure that did not repeat on retry",
            "summary": "single transient failure in {short}",
            "severity": "low",
            "action": "Record the failure and watch {short} for a repeat.",
        },
        {
            "text": "has a spelling mistake on one of its pages",
            "summary": "cosmetic text error in {short}",
            "severity": "low",
            "action": "Open a low-priority ticket to correct the text in {short}.",
        },
        {
            "text": "returned a result that looks wrong but has not been confirmed",
            "summary": "possible incorrect output from {short}",
            "severity": "medium",
            "action": "Reproduce the case against {short} and confirm before escalating.",
        },
        {
            "text": "is filling the disk faster than expected",
            "summary": "{short} consuming disk quickly",
            "severity": "medium",
            "action": "Check free space and the retention settings for {short}.",
        },
        {
            "text": "stopped without an error message and had to be restarted by hand",
            "summary": "{short} exited silently",
            "severity": "high",
            "action": "Page the on-call engineer and collect the logs from {short}.",
        },
    ],
    "contexts": [
        "since this morning's deploy",
        "for the last two hours",
        "intermittently since the weekend",
        "starting a few minutes ago",
        "every night this week",
    ],
    # Deliberately hard cases. A dataset without them trains a model that answers
    # confidently when the input does not support an answer.
    "edge_cases": [
        {
            "input": "Someone said something is broken. No other detail was given.",
            "summary": "report lacks any detail to triage",
            "severity": "low",
            "action": "Ask the reporter which system failed and when.",
        },
        {
            "input": "The dashboard is green and a colleague says everything looks fine today.",
            "summary": "no fault reported",
            "severity": "low",
            "action": "Close the report with no action.",
        },
        {
            "input": "Two reports arrived at once: the wiki is slow, and the backup job exited silently.",
            "summary": "two faults reported together",
            "severity": "high",
            "action": "Split the report and handle the backup job first.",
        },
        {
            "input": "A customer asks when the next release is. Nothing appears to be broken.",
            "summary": "question rather than a fault report",
            "severity": "low",
            "action": "Forward the question to the release owner.",
        },
    ],
}


def post_json(url: str, payload: dict, api_key: str | None, timeout: int) -> dict:
    """One OpenAI-compatible request. Same shape as Part 10's harness uses."""
    body = json.dumps(payload).encode("utf-8")
    headers = {"Content-Type": "application/json"}
    if api_key:
        headers["Authorization"] = f"Bearer {api_key}"
    request = urllib.request.Request(url, data=body, headers=headers, method="POST")
    try:
        with urllib.request.urlopen(request, timeout=timeout) as response:
            return json.loads(response.read().decode("utf-8"))
    except urllib.error.HTTPError as exc:
        detail = exc.read().decode("utf-8", "replace")[:300]
        raise RuntimeError(f"{url} returned HTTP {exc.code}: {detail}") from exc
    except urllib.error.URLError as exc:
        raise RuntimeError(f"cannot reach {url}: {exc.reason}") from exc


def rewrite_with_teacher(text: str, args, timeout: int = 120) -> str:
    """Ask a served model to make one report read like a person wrote it.

    The teacher never sees the answer and never writes one. It rewrites the question so
    that the training inputs are not all the same sentence with the nouns swapped, which
    is the one job here where a wrong output costs nothing: a badly rewritten question is
    still a question, whereas a badly written label is a wrong label.
    """
    payload = {
        "model": args.teacher_model,
        "messages": [
            {"role": "system", "content":
                "Rewrite the user's message as a short, natural report from a colleague. "
                "Keep every fact, keep it under 40 words, change nothing about what "
                "happened, and reply with the rewritten report only."},
            {"role": "user", "content": text},
        ],
        "temperature": 0.8,
        "max_tokens": 120,
    }
    data = post_json(f"{args.teacher_url.rstrip('/')}/chat/completions",
                     payload, args.teacher_key, timeout)
    content = (data.get("choices") or [{}])[0].get("message", {}).get("content", "")
    content = " ".join(str(content).split())
    return content or text


def build_examples(template: dict[str, Any], count: int, rng: random.Random) -> list[dict[str, str]]:
    """Every combination is drawn without replacement, so no two examples are identical."""
    combos = [
        (reporter, system, symptom, context)
        for reporter in template["reporters"]
        for system in template["systems"]
        for symptom in template["symptoms"]
        for context in template["contexts"]
    ]
    rng.shuffle(combos)
    edges = list(template["edge_cases"])
    wanted_edges = min(len(edges), max(1, count // 12))
    body = count - wanted_edges
    if body > len(combos):
        raise SystemExit(
            f"--count {count} needs {body} distinct combinations but the template only has "
            f"{len(combos)}. Add reporters, systems, symptoms or contexts, or ask for fewer."
        )

    examples: list[dict[str, str]] = []
    for reporter, system, symptom, context in combos[:body]:
        short = system["short"]
        examples.append({
            "input": f"{reporter} reports: {system['name']} {symptom['text']} {context}.",
            "summary": symptom["summary"].format(short=short),
            "severity": symptom["severity"],
            "action": symptom["action"].format(short=short),
            "kind": "generated",
        })
    for edge in edges[:wanted_edges]:
        examples.append({
            "input": edge["input"], "summary": edge["summary"],
            "severity": edge["severity"], "action": edge["action"], "kind": "edge",
        })
    rng.shuffle(examples)
    return examples


def answer_of(template: dict[str, Any], example: dict[str, str]) -> str:
    return template["answer_format"].format(
        summary=example["summary"], severity=example["severity"], action=example["action"])


def prompt_of(template: dict[str, Any], example: dict[str, str]) -> str:
    return f"{example['input']}\n\n{template['instruction']}"


def write_jsonl(path: Path, rows: list[dict[str, Any]]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with path.open("w", encoding="utf-8") as handle:
        for row in rows:
            handle.write(json.dumps(row, ensure_ascii=False) + "\n")


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--out-dir", default=".", help="directory to write the dataset into")
    parser.add_argument("--count", type=int, default=320, help="total examples before splitting")
    parser.add_argument("--valid-fraction", type=float, default=0.1)
    parser.add_argument("--task-count", type=int, default=24,
                        help="held-out examples turned into a Part 10 task file")
    parser.add_argument("--seed", type=int, default=0)
    parser.add_argument("--template", default=None, help="a JSON template from --print-template")
    parser.add_argument("--print-template", action="store_true",
                        help="print the built-in template and exit, so you can edit a copy")
    parser.add_argument("--teacher-url", default=None,
                        help="OpenAI-compatible base URL, e.g. http://127.0.0.1:8080/v1")
    parser.add_argument("--teacher-model", default=None, help="model name the endpoint answers to")
    parser.add_argument("--teacher-key", default=None, help="API key, if the endpoint needs one")
    parser.add_argument("--teacher-count", type=int, default=0,
                        help="how many inputs the teacher should rewrite; 0 disables it")
    args = parser.parse_args()

    if args.print_template:
        json.dump(TEMPLATE, sys.stdout, indent=2, ensure_ascii=False)
        print()
        return

    template = TEMPLATE
    if args.template:
        template = json.loads(Path(args.template).read_text(encoding="utf-8"))
        for key in ("system", "instruction", "answer_format", "reporters", "systems",
                    "symptoms", "contexts", "edge_cases"):
            if key not in template:
                raise SystemExit(f"{args.template} is missing the required key {key!r}")

    if args.teacher_count and not (args.teacher_url and args.teacher_model):
        raise SystemExit("--teacher-count needs both --teacher-url and --teacher-model")

    rng = random.Random(args.seed)
    examples = build_examples(template, args.count, rng)

    rewritten = 0
    for example in examples[: args.teacher_count]:
        if example["kind"] == "edge":
            continue  # the edge cases are the point; leave their wording alone
        try:
            example["input"] = rewrite_with_teacher(example["input"], args)
            example["kind"] = "generated-rewritten"
            rewritten += 1
        except RuntimeError as exc:
            print(f"teacher call failed, keeping the original wording: {exc}")
            break

    n_tasks = min(args.task_count, len(examples) // 4)
    held_out = examples[:n_tasks]
    remaining = examples[n_tasks:]
    n_valid = max(1, int(len(remaining) * args.valid_fraction))
    valid, train = remaining[:n_valid], remaining[n_valid:]

    out = Path(args.out_dir)
    system_message = {"role": "system", "content": template["system"]}

    def trl_rows(rows: list[dict[str, str]]) -> list[dict[str, Any]]:
        return [{
            "prompt": [system_message, {"role": "user", "content": prompt_of(template, r)}],
            "completion": [{"role": "assistant", "content": answer_of(template, r)}],
        } for r in rows]

    def mlx_rows(rows: list[dict[str, str]]) -> list[dict[str, Any]]:
        return [{"prompt": prompt_of(template, r), "completion": answer_of(template, r)}
                for r in rows]

    write_jsonl(out / "data" / "train.jsonl", trl_rows(train))
    write_jsonl(out / "data" / "valid.jsonl", trl_rows(valid))
    write_jsonl(out / "data-mlx" / "train.jsonl", mlx_rows(train))
    write_jsonl(out / "data-mlx" / "valid.jsonl", mlx_rows(valid))
    write_jsonl(out / "data-mlx" / "test.jsonl", mlx_rows(held_out))

    tasks = {
        "name": f"{template['name']}-format",
        "version": 1,
        "note": ("Held-out format tasks generated alongside the training data and never trained "
                 "on. Scored by Part 10's run-eval.py: the deterministic checks alone say whether "
                 "the output contract is being honoured, and the rubric lets a judge grade the "
                 "content of the three lines."),
        "settings": {"temperature": 0.0, "top_p": 1.0, "seed": 7, "max_tokens": 256,
                     "comment": "Fixed. The base model and the fine-tune are scored at these "
                                "same settings or the comparison means nothing."},
        "categories": {"format": "Does it produce the three required lines and nothing else?"},
        "tasks": [
            {
                "id": f"f{index + 1:02d}",
                "category": "format",
                "prompt": prompt_of(template, example),
                "reference": answer_of(template, example),
                "rubric": ("Exactly three lines, in the order Summary, Severity, Action, with no "
                           "preamble and no closing sentence. Severity is one of low, medium or "
                           "high and matches the seriousness of the report. The summary names "
                           "what failed. The action is a single imperative sentence."),
                "must_contain": template["must_contain"],
                "must_not_contain": template["must_not_contain"],
                "max_words": template["max_words"],
            }
            for index, example in enumerate(held_out)
        ],
    }
    (out / "format-tasks.json").write_text(
        json.dumps(tasks, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")

    (out / "format-dataset.json").write_text(json.dumps({
        "template": template["name"],
        "seed": args.seed,
        "count": len(examples),
        "teacher": {"url": args.teacher_url, "model": args.teacher_model,
                    "rewritten": rewritten} if rewritten else None,
        "splits": {"train": len(train), "validation": len(valid), "held_out_tasks": len(held_out)},
        "examples": examples,
    }, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")

    print(f"template:        {template['name']}")
    print(f"train:           {len(train)} example(s) -> {out / 'data' / 'train.jsonl'}")
    print(f"validation:      {len(valid)} example(s) -> {out / 'data' / 'valid.jsonl'}")
    print(f"held-out tasks:  {len(held_out)} -> {out / 'format-tasks.json'}")
    print(f"mlx-lm layout:   {out / 'data-mlx'}")
    if rewritten:
        print(f"teacher rewrote {rewritten} input(s); the answers were derived, not generated")
    print("\nRead ten training examples before you train on three hundred.")
    print("Then run decontaminate.py against your own Part 10 task file.")


if __name__ == "__main__":
    main()

Download make-format-dataset.py386 lines

The generator builds every example from structured fields, so the answers are correct by construction rather than by an author’s memory. It writes the TRL layout, the mlx-lm layout, and a held-out task file in the Part 10 harness’s shape that the training run never sees.

RunnableAll tracks

generate the dataset and the held-out task file

python3 make-format-dataset.py --out-dir . --count 320 --seed 0

Output — what you should see

template:        ticket-triage
train:           N example(s) -> data/train.jsonl
validation:      M example(s) -> data/valid.jsonl
held-out tasks:  24 -> format-tasks.json
mlx-lm layout:   data-mlx

To use your own format, print the template, edit the copy and pass it back. The vocabulary lists and the answer template are all in that one object, so replacing the format does not mean editing the generator.

RunnableAll tracks

take the template, edit it, and regenerate

python3 make-format-dataset.py --print-template > my-format.json
python3 make-format-dataset.py --template my-format.json --out-dir . --count 320

Optionally, put a served model to work rewriting the questions so that the training inputs do not all read like the same sentence with the nouns swapped. The teacher never writes an answer: the labels stay derived. Part 15 teaches generation properly, including how to use a teacher for the answers and how to check what you get.

RunnableAll tracks

optional: a local teacher rewrites the inputs, not the labels

python3 make-format-dataset.py --out-dir . --count 320 \
    --teacher-url http://127.0.0.1:8080/v1 \
    --teacher-model qwen3-8b \
    --teacher-count 80

Now read ten examples. Open data/train.jsonl and look at ten of them properly, including two of the edge cases. This is the step everyone skips and it is where every dataset problem is visible.

3. Check it against your own evaluation set

Copy the personal task set you froze in Part 10 before running the overlap check:

RunnableAll tracks

bring forward the regression set

cp "$LABS_ROOT/part-10-models-at-work/my-tasks.json" ./my-tasks.json

RunnableAll tracks

decontaminate.py

"""Check a supervised fine-tuning dataset for duplicates and for overlap with an evaluation set.

Purpose: the honesty check that has to run before any fine-tune is measured. Reads a
         training file and the Part 10 task file, reports exact duplicates inside the
         training data, exact matches against the evaluation set, and near-duplicates
         found by shared word n-grams, and can write a cleaned training file with the
         contaminated examples removed.
Platform: all (standard library only; no model, no accelerator, no network)
Minimum memory: 8 GB, and far less in practice: the whole check is text in memory
Assumes: Python 3.10 or newer. The training file is JSON Lines in any of the shapes
         Part 11's dataset lesson describes: {"prompt", "completion"}, {"messages": [...]}
         or {"text"}. The task file is the Part 10 evaluation set, a JSON object with a
         "tasks" list whose items carry "prompt" and usually "reference".

Usage: python3 decontaminate.py --train data/train.jsonl --tasks my-tasks.json
       python3 decontaminate.py --train data/train.jsonl --tasks my-tasks.json \
           --write-clean data/train-clean.jsonl --labbook labbook.md
       python3 decontaminate.py --train data/train.jsonl --valid data/valid.jsonl \
           --tasks my-tasks.json --n 13 --threshold 0.4 --strict

Two kinds of contamination are reported separately because they have different fixes.
An exact match means the same text is in both files, and the fix is to delete it from
the training data. A high n-gram containment means one text is largely contained in the
other, which is what happens when an evaluation prompt was paraphrased into a training
example, and the fix is a judgement call you have to make by reading the pair.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any

WORD_RE = re.compile(r"[a-z0-9]+")


def normalise(text: str) -> list[str]:
    """Lowercase, drop punctuation, split on words. Two texts that differ only in
    formatting have to compare equal, or the check misses the commonest duplicate."""
    return WORD_RE.findall(text.lower())


def ngrams(words: list[str], n: int) -> set[tuple[str, ...]]:
    """The set of word n-grams. Short texts fall back to one gram of the whole text so
    that a two-word answer is still comparable rather than silently empty."""
    if len(words) < n:
        return {tuple(words)} if words else set()
    return {tuple(words[i:i + n]) for i in range(len(words) - n + 1)}


def containment(a: set[tuple[str, ...]], b: set[tuple[str, ...]]) -> float:
    """How much of a is also in b, between 0 and 1. Containment rather than Jaccard,
    because a short evaluation prompt buried inside a long training example is exactly
    the case that matters and Jaccard would score it low."""
    if not a:
        return 0.0
    return len(a & b) / len(a)


def digest(words: list[str]) -> str:
    return hashlib.sha256(" ".join(words).encode("utf-8")).hexdigest()


def record_text(record: dict[str, Any]) -> str:
    """Flatten one training record to the text a comparison should see, whichever of the
    three dataset shapes it is in."""
    if "messages" in record:
        parts = []
        for message in record["messages"]:
            content = message.get("content")
            if isinstance(content, str):
                parts.append(content)
        return "\n".join(parts)
    if "prompt" in record or "completion" in record:
        parts = []
        for key in ("prompt", "completion"):
            value = record.get(key)
            if isinstance(value, str):
                parts.append(value)
            elif isinstance(value, list):
                for message in value:
                    content = message.get("content") if isinstance(message, dict) else None
                    if isinstance(content, str):
                        parts.append(content)
        return "\n".join(parts)
    if isinstance(record.get("text"), str):
        return record["text"]
    return ""


def load_jsonl(path: Path) -> list[dict[str, Any]]:
    records = []
    with path.open("r", encoding="utf-8") as handle:
        for number, line in enumerate(handle, start=1):
            line = line.strip()
            if not line:
                continue
            try:
                records.append(json.loads(line))
            except json.JSONDecodeError as exc:
                raise SystemExit(f"{path}:{number}: not valid JSON ({exc.msg})") from exc
    return records


def load_tasks(path: Path) -> list[dict[str, str]]:
    data = json.loads(path.read_text(encoding="utf-8"))
    tasks = data.get("tasks") if isinstance(data, dict) else data
    if not isinstance(tasks, list):
        raise SystemExit(f"{path} does not contain a list of tasks; expected the Part 10 task file")
    out = []
    for index, task in enumerate(tasks):
        text = "\n".join(
            str(task[key]) for key in ("prompt", "reference") if isinstance(task.get(key), str)
        )
        out.append({"id": str(task.get("id", f"task-{index:03d}")), "text": text})
    return out


def summarise(text: str, limit: int = 90) -> str:
    flat = " ".join(text.split())
    return flat if len(flat) <= limit else flat[: limit - 1] + "…"


def check_split(
    name: str,
    records: list[dict[str, Any]],
    task_grams: list[tuple[str, set[tuple[str, ...]]]],
    task_hashes: dict[str, str],
    n: int,
    threshold: float,
) -> dict[str, Any]:
    """One split against the evaluation set, plus its own internal duplicates."""
    seen: dict[str, int] = {}
    internal: list[dict[str, Any]] = []
    exact: list[dict[str, Any]] = []
    near: list[dict[str, Any]] = []
    contaminated: set[int] = set()
    empty = 0

    for index, record in enumerate(records):
        text = record_text(record)
        words = normalise(text)
        if not words:
            empty += 1
            continue
        key = digest(words)
        if key in seen:
            internal.append({"index": index, "duplicate_of": seen[key], "text": summarise(text)})
            contaminated.add(index)
            continue
        seen[key] = index

        if key in task_hashes:
            exact.append({"index": index, "task": task_hashes[key], "text": summarise(text)})
            contaminated.add(index)
            continue

        grams = ngrams(words, n)
        worst_task, worst_score = None, 0.0
        for task_id, gram_set in task_grams:
            score = max(containment(grams, gram_set), containment(gram_set, grams))
            if score > worst_score:
                worst_task, worst_score = task_id, score
        if worst_score >= threshold:
            near.append({
                "index": index, "task": worst_task,
                "containment": round(worst_score, 3), "text": summarise(text),
            })
            contaminated.add(index)

    return {
        "split": name,
        "examples": len(records),
        "empty_examples": empty,
        "internal_duplicates": internal,
        "exact_matches": exact,
        "near_duplicates": near,
        "contaminated_indices": sorted(contaminated),
    }


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--train", required=True, help="training file, JSON Lines")
    parser.add_argument("--valid", default=None, help="validation file, checked the same way")
    parser.add_argument("--tasks", required=True, help="the Part 10 evaluation task file")
    parser.add_argument("--n", type=int, default=13,
                        help="n-gram size for the near-duplicate check; 13 is the size the "
                             "decontamination sections of several pretraining papers use")
    parser.add_argument("--threshold", type=float, default=0.5,
                        help="report a pair when either text's n-grams are this fraction "
                             "contained in the other's")
    parser.add_argument("--report", default=None, help="write the full findings here as JSON")
    parser.add_argument("--write-clean", default=None,
                        help="write the training file with every flagged example removed")
    parser.add_argument("--labbook", default=None, help="append one JSON line recording this check")
    parser.add_argument("--strict", action="store_true",
                        help="exit 1 when anything was flagged, for use in a script")
    args = parser.parse_args()

    train_path = Path(args.train)
    tasks_path = Path(args.tasks)
    for path in (train_path, tasks_path):
        if not path.is_file():
            raise SystemExit(f"{path} does not exist")

    tasks = load_tasks(tasks_path)
    task_grams = [(t["id"], ngrams(normalise(t["text"]), args.n)) for t in tasks]
    task_hashes = {digest(normalise(t["text"])): t["id"] for t in tasks}

    train_records = load_jsonl(train_path)
    results = [check_split("train", train_records, task_grams, task_hashes, args.n, args.threshold)]
    if args.valid:
        valid_path = Path(args.valid)
        if not valid_path.is_file():
            raise SystemExit(f"{valid_path} does not exist")
        results.append(check_split("validation", load_jsonl(valid_path), task_grams,
                                   task_hashes, args.n, args.threshold))

    flagged = 0
    print(f"tasks in evaluation set: {len(tasks)}   n-gram size: {args.n}   "
          f"threshold: {args.threshold}")
    for result in results:
        counts = (len(result["internal_duplicates"]), len(result["exact_matches"]),
                  len(result["near_duplicates"]))
        flagged += sum(counts)
        print(f"\n{result['split']}: {result['examples']} example(s), "
              f"{counts[0]} internal duplicate(s), {counts[1]} exact match(es) against the "
              f"evaluation set, {counts[2]} near-duplicate(s)")
        for row in result["exact_matches"][:5]:
            print(f"  EXACT  line {row['index'] + 1} == task {row['task']}: {row['text']}")
        for row in result["near_duplicates"][:5]:
            print(f"  NEAR   line {row['index'] + 1} ~ task {row['task']} "
                  f"(containment {row['containment']}): {row['text']}")
        for row in result["internal_duplicates"][:5]:
            print(f"  DUP    line {row['index'] + 1} repeats line {row['duplicate_of'] + 1}: "
                  f"{row['text']}")

    if args.report:
        Path(args.report).write_text(
            json.dumps({"tasks_file": str(tasks_path), "n": args.n,
                        "threshold": args.threshold, "results": results}, indent=2),
            encoding="utf-8")
        print(f"\nfull findings written to {args.report}")

    if args.write_clean:
        drop = set(results[0]["contaminated_indices"])
        kept = [r for i, r in enumerate(train_records) if i not in drop]
        with Path(args.write_clean).open("w", encoding="utf-8") as handle:
            for record in kept:
                handle.write(json.dumps(record, ensure_ascii=False) + "\n")
        print(f"clean training file: {args.write_clean} "
              f"({len(kept)} kept, {len(train_records) - len(kept)} removed)")

    if args.labbook:
        line = {
            "check": "part-13/decontaminate",
            "date": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
            "train": str(train_path),
            "tasks": str(tasks_path),
            "n": args.n,
            "threshold": args.threshold,
            "flagged": flagged,
            "per_split": [
                {"split": r["split"], "examples": r["examples"],
                 "internal_duplicates": len(r["internal_duplicates"]),
                 "exact_matches": len(r["exact_matches"]),
                 "near_duplicates": len(r["near_duplicates"])}
                for r in results
            ],
        }
        book = Path(args.labbook)
        if not book.exists():
            book.write_text("# Lab notebook\n\n## Results\n\n", encoding="utf-8")
        with book.open("a", encoding="utf-8") as handle:
            handle.write(json.dumps(line, sort_keys=True) + "\n")
        print(f"recorded the check in {args.labbook}")

    if flagged == 0:
        print("\nNothing flagged. The training data and the evaluation set are disjoint at "
              "this n-gram size and threshold, which is what a comparable measurement needs.")
    else:
        print(f"\n{flagged} item(s) flagged. Read them before you train: a number measured on a "
              "contaminated evaluation set is not a measurement.")
    if args.strict and flagged:
        sys.exit(1)


if __name__ == "__main__":
    main()

Download decontaminate.py294 lines

RunnableAll tracks

deduplicate, and check for overlap with your Part 10 tasks

python3 decontaminate.py \
    --train data/train.jsonl \
    --valid data/valid.jsonl \
    --tasks my-tasks.json \
    --report decontamination-report.json \
    --labbook labbook.md

For a generated dataset this should find nothing, and running it anyway is the point: you are establishing that the regression check in task 6 is measuring something. When you repeat this lab with data of your own, this is the step that stops a contaminated number reaching your notebook.

4. Train

RunnableAll tracks

sftlog.py

"""Append one machine-readable record per fine-tuning run or evaluation to the lab notebook.

Purpose: Part 13's self-contained copy of the run-log format defined in Part 11, so that
         this part's scripts record a run identically without depending on Part 11's files
         being on the path. Every field is filled in or written as null, because a reader a
         month later has to be able to tell "not recorded" from "not applicable".
Platform: all (standard library only; torch, transformers, trl, peft and mlx are inspected
          for their version strings only if they happen to be installed)
Minimum memory: 8 GB
Assumes: Python 3.10 or newer. The lab notebook is created if it does not exist. git is
         optional and is used only to record the commit the configuration was at.

Usage: imported by this part's Python scripts:
           import sftlog; sftlog.record(labbook="labbook.md", lab="part-13/train-lora", ...)
       or called from a shell script with the run's own fields as JSON on stdin:
           python3 sftlog.py --record --labbook labbook.md < fields.json
       or run with no arguments to print the field list and exit.
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import platform
import secrets
import shutil
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any

# The same field set Part 11 defines. A record missing any of them is refused, because a
# partial record is harder to interpret than no record at all.
FIELDS = (
    "run_id", "lab", "date", "config_commit", "model", "dataset",
    "hyperparameters", "seed", "hardware", "versions", "losses", "scores", "notes",
)

PACKAGES = ("torch", "transformers", "trl", "peft", "datasets", "bitsandbytes", "mlx", "mlx-lm")


def new_run_id() -> str:
    """Sorts by time and does not collide between two runs started in the same second."""
    stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
    return f"{stamp}-{secrets.token_hex(3)}"


def file_sha256(path: str | os.PathLike[str], chunk: int = 1 << 20) -> str | None:
    """Content hash of a dataset file, so a run is tied to the exact bytes it read.

    This is the field people leave out and then need: a month later the question is not
    which learning rate was used, which is in the script, but whether this run was before
    or after the dataset was fixed.
    """
    p = Path(path)
    if not p.is_file():
        return None
    digest = hashlib.sha256()
    with p.open("rb") as handle:
        while True:
            block = handle.read(chunk)
            if not block:
                break
            digest.update(block)
    return digest.hexdigest()


def git_commit(path: str | os.PathLike[str] = ".") -> str | None:
    """The commit the configuration is at, with -dirty when the tree had uncommitted edits."""
    if shutil.which("git") is None:
        return None
    target = Path(path)
    cwd = target if target.is_dir() else target.parent
    try:
        rev = subprocess.run(["git", "rev-parse", "--short", "HEAD"], cwd=cwd,
                             capture_output=True, text=True, check=True, timeout=10).stdout.strip()
        dirty = subprocess.run(["git", "status", "--porcelain"], cwd=cwd,
                               capture_output=True, text=True, check=True, timeout=10).stdout.strip()
    except (subprocess.SubprocessError, OSError):
        return None
    return f"{rev}-dirty" if dirty else rev


def describe_hardware() -> dict[str, Any]:
    """What the run executed on, as far as it can be established without extra packages."""
    info: dict[str, Any] = {
        "os": f"{platform.system()} {platform.release()}",
        "machine": platform.machine(),
        "python": platform.python_version(),
        "accelerator": "cpu",
        "device_name": None,
    }
    try:
        import torch  # noqa: PLC0415 - optional, and only for reporting
    except ImportError:
        return info
    if torch.cuda.is_available():
        info["accelerator"] = "cuda"
        info["device_name"] = torch.cuda.get_device_name(0)
        return info
    mps = getattr(torch.backends, "mps", None)
    if mps is not None and mps.is_available():
        info["accelerator"] = "mps"
        info["device_name"] = platform.processor() or "Apple silicon"
    return info


def package_versions(names: tuple[str, ...] = PACKAGES) -> dict[str, str | None]:
    """Version strings for the packages that decide what a run actually did."""
    from importlib.metadata import PackageNotFoundError, version  # noqa: PLC0415

    out: dict[str, str | None] = {}
    for name in names:
        try:
            out[name] = version(name)
        except PackageNotFoundError:
            out[name] = None
    return out


def build_record(
    lab: str,
    model: str,
    dataset: dict[str, Any],
    hyperparameters: dict[str, Any],
    seed: int,
    losses: dict[str, Any] | None = None,
    scores: dict[str, Any] | None = None,
    config_path: str | os.PathLike[str] | None = None,
    notes: str | None = None,
) -> dict[str, Any]:
    """Assemble the record. Separate from writing so a caller can inspect it first."""
    return {
        "run_id": new_run_id(),
        "lab": lab,
        "date": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
        "config_commit": git_commit(config_path if config_path is not None else "."),
        "model": model,
        "dataset": dataset,
        "hyperparameters": hyperparameters,
        "seed": seed,
        "hardware": describe_hardware(),
        "versions": package_versions(),
        "losses": losses or {},
        "scores": scores or {},
        "notes": notes,
    }


def append(record: dict[str, Any], labbook: str | os.PathLike[str] = "labbook.md") -> Path:
    """Append one JSON line. A missing required field is an error, not a warning."""
    missing = [f for f in FIELDS if f not in record]
    if missing:
        raise ValueError(f"run record is missing required fields: {', '.join(missing)}")
    path = Path(labbook)
    if not path.exists():
        path.write_text("# Lab notebook\n\n## Results\n\n", encoding="utf-8")
    with path.open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(record, sort_keys=True) + "\n")
    return path


def record(labbook: str | os.PathLike[str] = "labbook.md", **kwargs: Any) -> dict[str, Any]:
    """Build and append in one call; returns the record so a caller can print its id."""
    rec = build_record(**kwargs)
    append(rec, labbook)
    return rec


def main() -> None:
    parser = argparse.ArgumentParser(description="Run-log helper for Part 13's fine-tuning labs.")
    parser.add_argument("--record", action="store_true",
                        help="read this run's own fields as a JSON object on stdin")
    parser.add_argument("--labbook", default="labbook.md")
    args = parser.parse_args()

    if not args.record:
        print(__doc__)
        print("Fields in every record:", ", ".join(FIELDS))
        return

    fields = json.load(sys.stdin)
    allowed = {"lab", "model", "dataset", "hyperparameters", "seed",
               "losses", "scores", "config_path", "notes"}
    unknown = set(fields) - allowed
    if unknown:
        raise SystemExit(f"unknown field(s) on stdin: {', '.join(sorted(unknown))}")
    rec = record(labbook=args.labbook, **fields)
    print(f"recorded run {rec['run_id']} in {args.labbook}")


if __name__ == "__main__":
    main()

Download sftlog.py195 lines

RunnableAll tracks

train-lora.py

"""Train a LoRA adapter on a format-following dataset with TRL's SFTTrainer and PEFT.

Purpose: Part 13's self-contained supervised fine-tuning run for Tracks S, X and N. Loads the
         conversational prompt-completion files that make-format-dataset.py wrote, attaches a
         LoRA adapter, trains in bfloat16 with an evaluation after every epoch, stops early
         when the evaluation loss stops improving, keeps the best-scoring checkpoint, and
         appends one run record to the lab notebook.
Platform: spark, strix, nvidia (CUDA, or ROCm which also reports as cuda to PyTorch). It runs
          on the CPU too, slowly. Track M uses train-lora-mlx.sh instead: PyTorch's MPS
          backend will run this in float32, but mlx-lm is the supported Mac path.
Minimum memory: 12 GB for a 1.7B to 4B base at bfloat16 with a rank-16 adapter, batch 1 and
          a 1,024-token sequence. Part 11's memory lesson has the arithmetic.
Assumes: torch, transformers, trl, peft and datasets installed in the active environment;
         make-format-dataset.py has been run so that data/train.jsonl and data/valid.jsonl
         exist; sftlog.py sits next to this file.

Usage: python3 train-lora.py --model Qwen/Qwen3-1.7B --data-dir data \
           --output-dir runs/format-qwen3-1.7b --labbook labbook.md
       python3 train-lora.py --model Qwen/Qwen3-4B --list-modules
       python3 train-lora.py --model Qwen/Qwen3-4B --epochs 3 --rank 32 --alpha 64 \
           --gradient-checkpointing --output-dir runs/format-qwen3-4b
"""
from __future__ import annotations

import argparse
import json
import time
from pathlib import Path

import torch
from datasets import load_dataset
from peft import LoraConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, EarlyStoppingCallback
from trl import SFTConfig, SFTTrainer

import sftlog

# The seven linear projections of a Qwen3 block: four in attention, three in the
# feed-forward network. --list-modules prints what your own base model actually has,
# because a name that does not match attaches nothing and raises no error.
DEFAULT_TARGETS = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]


def pick_device() -> str:
    """CUDA (or ROCm, which reports as cuda), then MPS, then CPU. Same choice as Part 11."""
    if torch.cuda.is_available():
        return "cuda"
    mps = getattr(torch.backends, "mps", None)
    if mps is not None and mps.is_available():
        return "mps"
    return "cpu"


def use_bf16(device: str, requested: str) -> bool:
    """bfloat16 only where the device supports it; everything else trains in float32."""
    if requested == "fp32":
        return False
    if requested == "bf16":
        return True
    return device == "cuda" and torch.cuda.is_bf16_supported()


def list_linear_modules(model_id: str) -> None:
    """Print the names LoRA can target, so target_modules is never guessed."""
    model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float32)
    names = sorted({name.split(".")[-1] for name, module in model.named_modules()
                    if isinstance(module, torch.nn.Linear)})
    print(f"linear module names in {model_id}:")
    for name in names:
        print(f"  {name}")
    print("\nPass the ones you want with --target-modules, or use --target-modules all-linear.")


def summarise_history(history: list[dict]) -> dict[str, float | int | None]:
    """The five numbers worth keeping out of a log full of them."""
    train_losses = [row["loss"] for row in history if "loss" in row]
    evals = [(row.get("epoch"), row["eval_loss"]) for row in history if "eval_loss" in row]
    best_epoch, best_eval = min(evals, key=lambda pair: pair[1]) if evals else (None, None)
    return {
        "first_train_loss": round(train_losses[0], 4) if train_losses else None,
        "final_train_loss": round(train_losses[-1], 4) if train_losses else None,
        "final_eval_loss": round(evals[-1][1], 4) if evals else None,
        "best_eval_loss": round(best_eval, 4) if best_eval is not None else None,
        "best_epoch": best_epoch,
        "evaluations": len(evals),
    }


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--model", default="Qwen/Qwen3-1.7B",
                        help="base model repository id or local path; use an instruct "
                             "checkpoint, which already carries a chat template")
    parser.add_argument("--data-dir", default="data",
                        help="directory holding train.jsonl and valid.jsonl")
    parser.add_argument("--output-dir", default="runs/format-lora")
    parser.add_argument("--epochs", type=float, default=3.0)
    parser.add_argument("--batch-size", type=int, default=1, help="per-device batch size")
    parser.add_argument("--grad-accum", type=int, default=8,
                        help="batches summed before one optimiser step; only --batch-size "
                             "costs memory, so this is how you raise the effective batch")
    parser.add_argument("--lr", type=float, default=1e-4,
                        help="adapters take roughly 1e-4, not the SFTConfig default of 2e-5")
    parser.add_argument("--max-length", type=int, default=1024)
    parser.add_argument("--rank", type=int, default=16)
    parser.add_argument("--alpha", type=int, default=32)
    parser.add_argument("--dropout", type=float, default=0.05)
    parser.add_argument("--target-modules", nargs="+", default=DEFAULT_TARGETS,
                        help='module names, or the single word all-linear')
    parser.add_argument("--use-dora", action="store_true",
                        help="decompose the update into magnitude and direction; helps most "
                             "at low rank, and makes each step slower")
    parser.add_argument("--use-rslora", action="store_true",
                        help="scale by alpha over the square root of the rank; for unstable "
                             "high-rank runs")
    parser.add_argument("--precision", choices=["auto", "bf16", "fp32"], default="auto")
    parser.add_argument("--gradient-checkpointing", action="store_true",
                        help="recompute activations in the backward pass: saves memory, costs time")
    parser.add_argument("--early-stopping-patience", type=int, default=2,
                        help="stop after this many evaluations without an improvement; 0 disables")
    parser.add_argument("--seed", type=int, default=0)
    parser.add_argument("--labbook", default=None, help="append one JSON run record to this file")
    parser.add_argument("--notes", default=None, help="one line about what this run is testing")
    parser.add_argument("--list-modules", action="store_true",
                        help="print the base model's linear module names and exit")
    args = parser.parse_args()

    if args.list_modules:
        list_linear_modules(args.model)
        return

    device = pick_device()
    bf16 = use_bf16(device, args.precision)
    dtype = torch.bfloat16 if bf16 else torch.float32
    print(f"device: {device}   precision: {'bfloat16' if bf16 else 'float32'}")
    if device == "mps":
        print("note: this is Track M's fallback path in float32. train-lora-mlx.sh is faster "
              "and is the path the lab describes for a Mac.")

    data_dir = Path(args.data_dir)
    files = {"train": str(data_dir / "train.jsonl"), "validation": str(data_dir / "valid.jsonl")}
    for split, path in files.items():
        if not Path(path).is_file():
            raise SystemExit(f"{path} is missing ({split} split); run make-format-dataset.py first")
    dataset = load_dataset("json", data_files=files)
    print(f"train examples: {len(dataset['train'])}   "
          f"validation examples: {len(dataset['validation'])}")

    tokenizer = AutoTokenizer.from_pretrained(args.model)
    if tokenizer.chat_template is None:
        raise SystemExit(
            f"{args.model} has no chat template, so it is a base checkpoint rather than an "
            "instruct one. Either pick an instruct model or set chat_template_path in SFTConfig."
        )

    targets = args.target_modules[0] if args.target_modules == ["all-linear"] else args.target_modules

    config = SFTConfig(
        output_dir=args.output_dir,
        num_train_epochs=args.epochs,
        per_device_train_batch_size=args.batch_size,
        per_device_eval_batch_size=args.batch_size,
        gradient_accumulation_steps=args.grad_accum,
        learning_rate=args.lr,
        lr_scheduler_type="cosine",
        warmup_steps=5,
        max_length=args.max_length,
        packing=False,
        completion_only_loss=True,   # the loss lands on the answer, not on your questions
        gradient_checkpointing=args.gradient_checkpointing,
        bf16=bf16,
        model_init_kwargs={"dtype": dtype},
        eval_strategy="epoch",
        save_strategy="epoch",       # must match eval_strategy for the next line to work
        save_total_limit=2,
        load_best_model_at_end=True,
        metric_for_best_model="eval_loss",
        greater_is_better=False,
        logging_steps=5,
        report_to="none",
        seed=args.seed,
        data_seed=args.seed,
    )

    peft_config = LoraConfig(
        r=args.rank,
        lora_alpha=args.alpha,
        lora_dropout=args.dropout,
        target_modules=targets,
        use_dora=args.use_dora,
        use_rslora=args.use_rslora,
        bias="none",
        task_type="CAUSAL_LM",
    )

    callbacks = []
    if args.early_stopping_patience > 0:
        callbacks.append(EarlyStoppingCallback(early_stopping_patience=args.early_stopping_patience))

    trainer = SFTTrainer(
        model=args.model,
        args=config,
        train_dataset=dataset["train"],
        eval_dataset=dataset["validation"],
        processing_class=tokenizer,
        peft_config=peft_config,
        callbacks=callbacks or None,
    )
    # If this percentage is not roughly what the adapter arithmetic predicted, the target
    # module names did not match and nothing was attached.
    trainer.model.print_trainable_parameters()

    started = time.time()
    trainer.train()
    elapsed = time.time() - started

    trainer.save_model(args.output_dir)
    # The chat template travels in the tokeniser files, and it is the thing that has to
    # match at serving time. Saving it here is not optional.
    tokenizer.save_pretrained(args.output_dir)

    losses = summarise_history(trainer.state.log_history)
    losses["seconds"] = round(elapsed, 1)
    print(json.dumps(losses, indent=2))
    print(f"adapter saved to {args.output_dir}")
    if losses["best_epoch"] is not None and losses["best_epoch"] <= 1:
        print("the best epoch was the first: this dataset is small for this many epochs, or "
              "the learning rate is too high. Read the two curves before you train again.")

    if args.labbook:
        record = sftlog.record(
            labbook=args.labbook,
            lab="part-13/train-lora",
            model=args.model,
            dataset={
                "path": files["train"],
                "sha256": sftlog.file_sha256(files["train"]),
                "train_examples": len(dataset["train"]),
                "validation_examples": len(dataset["validation"]),
            },
            hyperparameters={
                "method": "dora" if args.use_dora else "lora",
                "rank": args.rank,
                "alpha": args.alpha,
                "dropout": args.dropout,
                "target_modules": targets,
                "use_rslora": args.use_rslora,
                "epochs": args.epochs,
                "batch_size": args.batch_size,
                "grad_accum": args.grad_accum,
                "effective_batch": args.batch_size * args.grad_accum,
                "learning_rate": args.lr,
                "max_length": args.max_length,
                "gradient_checkpointing": args.gradient_checkpointing,
                "early_stopping_patience": args.early_stopping_patience,
                "precision": "bfloat16" if bf16 else "float32",
                "completion_only_loss": True,
                "output_dir": args.output_dir,
            },
            seed=args.seed,
            losses=losses,
            scores={},
            config_path=__file__,
            notes=args.notes,
        )
        print(f"recorded run {record['run_id']} in {args.labbook}")


if __name__ == "__main__":
    main()

Download train-lora.py270 lines

Before the first run, print the base model’s linear module names, so that --target-modules is something you checked rather than something you copied.

RunnableAll tracks

what can this model's adapters attach to?

python3 train-lora.py --model Qwen/Qwen3-1.7B --list-modules

Track S — NVIDIA DGX Spark

RunnableTrack S · DGX Spark

the training run

python3 train-lora.py \
    --model Qwen/Qwen3-4B \
    --data-dir data \
    --output-dir runs/format-qwen3-4b \
    --epochs 3 \
    --batch-size 2 \
    --grad-accum 4 \
    --lr 1e-4 \
    --rank 16 \
    --alpha 32 \
    --labbook labbook.md \
    --notes "ticket triage format, first run"

Track X — AMD Ryzen AI Max+ 395

RunnableTrack X · Ryzen AI Max+

the training run on ROCm PyTorch

python3 train-lora.py \
    --model Qwen/Qwen3-1.7B \
    --data-dir data \
    --output-dir runs/format-qwen3-1.7b \
    --epochs 3 \
    --batch-size 1 \
    --grad-accum 8 \
    --lr 1e-4 \
    --gradient-checkpointing \
    --labbook labbook.md \
    --notes "ticket triage format, ROCm, first run"

The device line should say cuda, because that is how PyTorch reports a ROCm device. If it says cpu, the ROCm build is not the one in your active environment and Part 11’s environment lesson is where to go.

Track M — Apple silicon

RunnableTrack M · Apple silicon

train-lora-mlx.sh

#!/usr/bin/env bash
# Purpose: Track M's LoRA fine-tune. Train an adapter with mlx_lm.lora on the dataset
#          make-format-dataset.py wrote, report the held-out loss and perplexity, fuse the
#          adapter into a standalone model, generate one answer to see the format, and
#          append a run record to the lab notebook
# Platform: mac (Apple silicon, MLX). Tracks S, X and N use train-lora.py instead
# Minimum memory: 12 GB for a 1.7B to 4B base; --num-layers is the memory knob if it is tight
# Assumes: mlx-lm installed in the active environment so that mlx_lm.lora, mlx_lm.fuse and
#          mlx_lm.generate are on PATH; make-format-dataset.py has been run so that
#          data-mlx/{train,valid,test}.jsonl exist; sftlog.py sits next to this script;
#          run from the directory holding data-mlx
#
# Usage: bash train-lora-mlx.sh [MODEL] [ITERS]
#   MODEL  defaults to the MLX community conversion of Qwen3-1.7B
#   ITERS  defaults to 600
#
# Environment: DATA, ADAPTERS, FUSED, BATCH_SIZE, NUM_LAYERS, LEARNING_RATE, FINE_TUNE_TYPE
#              (lora, dora or full), SEED, LABBOOK, SKIP_FUSE=1 to stop after training.
set -euo pipefail

MODEL="${1:-mlx-community/Qwen3-1.7B-bf16}"
ITERS="${2:-600}"
DATA="${DATA:-data-mlx}"
ADAPTERS="${ADAPTERS:-runs/format-mlx-adapters}"
FUSED="${FUSED:-models/format-mlx-fused}"
BATCH_SIZE="${BATCH_SIZE:-2}"
NUM_LAYERS="${NUM_LAYERS:-16}"
LEARNING_RATE="${LEARNING_RATE:-1e-4}"
FINE_TUNE_TYPE="${FINE_TUNE_TYPE:-lora}"
SEED="${SEED:-0}"
LABBOOK="${LABBOOK:-labbook.md}"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

die() { echo "train-lora-mlx: $*" >&2; exit 1; }

command -v mlx_lm.lora >/dev/null || die "mlx_lm.lora is not on PATH; install mlx-lm in the active environment"
command -v python3 >/dev/null || die "python3 is not on PATH"
[[ -f "$HERE/sftlog.py" ]] || die "sftlog.py is not next to this script"
for split in train valid test; do
  [[ -f "$DATA/$split.jsonl" ]] || die "$DATA/$split.jsonl is missing; run: python3 make-format-dataset.py --out-dir ."
done

TRAIN_N=$(wc -l < "$DATA/train.jsonl" | tr -d ' ')
VALID_N=$(wc -l < "$DATA/valid.jsonl" | tr -d ' ')
echo "==> Training a $FINE_TUNE_TYPE adapter on $MODEL"
echo "    $TRAIN_N training example(s), $VALID_N validation example(s), $ITERS iterations"
echo "    --mask-prompt puts the loss on the completion only, which is what"
echo "    completion_only_loss does on the other three tracks."

mkdir -p "$(dirname "$ADAPTERS")" "$(dirname "$FUSED")"
START=$(date +%s)
mlx_lm.lora \
  --model "$MODEL" \
  --train \
  --data "$DATA" \
  --iters "$ITERS" \
  --batch-size "$BATCH_SIZE" \
  --num-layers "$NUM_LAYERS" \
  --learning-rate "$LEARNING_RATE" \
  --fine-tune-type "$FINE_TUNE_TYPE" \
  --mask-prompt \
  --adapter-path "$ADAPTERS"
ELAPSED=$(( $(date +%s) - START ))

echo "==> Held-out loss and perplexity, with the adapter attached"
echo "    This is the test.jsonl split, which training never saw. It is the MLX"
echo "    equivalent of the evaluation loss the other tracks print each epoch."
mlx_lm.lora \
  --model "$MODEL" \
  --data "$DATA" \
  --adapter-path "$ADAPTERS" \
  --test | tee "$ADAPTERS/test.txt"

echo "==> One answer, to see whether the format was learned"
mlx_lm.generate \
  --model "$MODEL" \
  --adapter-path "$ADAPTERS" \
  --max-tokens 96 \
  --temp 0 \
  --seed 0 \
  --prompt "The monitoring system reports: the model gateway is unreachable from every machine we have tried this morning.

Triage this report. Reply with exactly three lines and nothing else:
Summary: one line, under twelve words
Severity: one of low, medium or high
Action: one imperative sentence"

if [[ "${SKIP_FUSE:-0}" == "1" ]]; then
  echo "==> Skipped fusing (SKIP_FUSE=1)"
else
  echo "==> Fusing the adapter into a standalone model at $FUSED"
  echo "    The fused directory is what llama.cpp's convert_hf_to_gguf.py reads if you"
  echo "    want a GGUF as well. mlx_lm.fuse can write GGUF directly with --export-gguf,"
  echo "    but that path covers a narrower set of architectures."
  mlx_lm.fuse \
    --model "$MODEL" \
    --adapter-path "$ADAPTERS" \
    --save-path "$FUSED"
fi

echo "==> Recording the run in $LABBOOK"
DATA_SHA=$(shasum -a 256 "$DATA/train.jsonl" | cut -d' ' -f1)
python3 "$HERE/sftlog.py" --record --labbook "$LABBOOK" <<JSON
{
  "lab": "part-13/train-lora-mlx",
  "model": "$MODEL",
  "dataset": {"path": "$DATA/train.jsonl", "sha256": "$DATA_SHA",
              "train_examples": $TRAIN_N, "validation_examples": $VALID_N},
  "hyperparameters": {"method": "$FINE_TUNE_TYPE", "iters": $ITERS,
                      "batch_size": $BATCH_SIZE, "num_layers": $NUM_LAYERS,
                      "learning_rate": $LEARNING_RATE, "mask_prompt": true,
                      "adapter_path": "$ADAPTERS", "fused_path": "$FUSED",
                      "seconds": $ELAPSED},
  "seed": $SEED,
  "losses": {},
  "scores": {},
  "notes": "copy the final train and validation loss and the test perplexity out of the mlx_lm.lora output above into the losses field"
}
JSON

echo
echo "==> Done in ${ELAPSED}s."
echo "    adapter:      $ADAPTERS"
echo "    test output:  $ADAPTERS/test.txt"
echo "    fused model:  $FUSED"
echo "    Next: serve it and score it against the base with evaluate-against-base.py."

Download train-lora-mlx.sh126 lines

RunnableTrack M · Apple silicon

the training run with mlx-lm

bash train-lora-mlx.sh mlx-community/Qwen3-1.7B-bf16 600

--num-layers defaults to 16 in the script as it does in the tool. Lower it with NUM_LAYERS=8 bash train-lora-mlx.sh if memory is tight, and record that you did: fewer adapted layers is less capacity, and it belongs in the comparison rather than in your memory.

Track N — NVIDIA desktop or laptop

RunnableTrack N · NVIDIA GPU

the training run

python3 train-lora.py \
    --model Qwen/Qwen3-1.7B \
    --data-dir data \
    --output-dir runs/format-qwen3-1.7b \
    --epochs 3 \
    --batch-size 1 \
    --grad-accum 8 \
    --lr 1e-4 \
    --gradient-checkpointing \
    --labbook labbook.md \
    --notes "ticket triage format, first run"

On 16 GB and above, swap the model for Qwen/Qwen3-4B and raise --batch-size to 2 with --grad-accum 4, which keeps the effective batch at 8.

The first thing the script prints after loading is the trainable-parameter count. Check it against the arithmetic from the second lesson: rank times the sum of the input and output widths, summed over every adapted layer, should be well under one per cent of the model. A number far below that means the module names did not match.

5. Read the two curves before you believe anything

The run prints a summary when it finishes: the first and last training loss, the last and best evaluation loss, and which epoch was best.

Output — what you should see

{
  "first_train_loss": ...,
  "final_train_loss": ...,
  "final_eval_loss": ...,
  "best_eval_loss": ...,
  "best_epoch": ...,
  "evaluations": 3,
  "seconds": ...
}

Three readings, and each one changes what you do next. The best epoch in the middle is the healthy case: train and evaluation losses both fell, then the evaluation loss flattened, and the run kept the right checkpoint. The best epoch is the first means three epochs was too many for this much data, and the script says so; drop to two, or generate more examples. Neither loss moved means the learning rate is too low or the adapter attached to nothing, and the trainable-parameter count from task 4 tells you which.

=Expected result

On the 1.7-billion-parameter base with a few hundred short examples, this run finishes in minutes rather than hours on every track, and the training loss falls steeply over the first few dozen steps before flattening. If it is still running after half an hour on an accelerator, check the device line the script printed: a run that fell back to the CPU is the usual explanation.

6. Export the model you actually trained

Use the checkpoint from task 4, not a similarly named model from another run. Track S’s command trained the 4B base; Tracks X and N trained 1.7B; Track M produced an MLX adapter and fused model. Do not attach an adapter from one base to another. Export before comparing so you evaluate the representation you intend to serve.

RunnableAll tracks

merge-adapter.py

"""Merge a LoRA adapter into its base model at bfloat16, and prove the merge did not change it.

Purpose: turn the adapter directory a training run produced into an ordinary model that can be
         converted to GGUF, converted to MLX or served directly. Loads the base at bfloat16 on
         the CPU, merges, saves the merged model together with the tokeniser that carries the
         chat template, and optionally generates the same prompts through the unmerged and the
         merged model to show that they agree.
Platform: all (the merge is weight arithmetic, not a forward pass, so it needs no accelerator;
          the optional comparison runs on whatever device is available). Track M merges MLX
          adapters with mlx_lm.fuse instead, which the lab describes.
Minimum memory: enough system memory for one bfloat16 copy of the base model, so about 4 GB for
          a 1.7B model and about 8 GB for a 4B one, plus room for the comparison if you use it
Assumes: torch, transformers and peft installed; the adapter directory contains
         adapter_config.json naming the base model it was trained against.

Usage: python3 merge-adapter.py --adapter runs/format-qwen3-1.7b
       python3 merge-adapter.py --adapter runs/format-qwen3-4b \
           --merged-dir models/format-qwen3-4b-merged --compare 4
       python3 merge-adapter.py --adapter runs/format-qwen3-4b --base-override Qwen/Qwen3-4B

Merging into a base that is not the one the adapter was trained against, or into a quantised
base, produces a model that loads and runs and is wrong. The default is to use the base named
in adapter_config.json; --base-override exists for the case where you moved the base on disk,
and the script tells you loudly when you use it.
"""
from __future__ import annotations

import argparse
import json
from pathlib import Path

import torch
from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_PROMPTS = [
    "The monitoring system reports: the model gateway is unreachable from every machine "
    "we have tried since this morning's deploy.",
    "A customer asks when the next release is. Nothing appears to be broken.",
    "The overnight operator reports: the nightly backup job is filling the disk faster "
    "than expected every night this week.",
    "Someone said something is broken. No other detail was given.",
]


def generate(model, tokenizer, prompt: str, instruction: str, max_new_tokens: int) -> str:
    """One greedy generation, so two models given the same input can be compared exactly."""
    messages = [{"role": "user", "content": f"{prompt}\n\n{instruction}" if instruction else prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt", add_special_tokens=False).to(model.device)
    with torch.no_grad():
        output = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)
    return tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--adapter", required=True, help="directory saved by train-lora.py")
    parser.add_argument("--merged-dir", default=None,
                        help="where to write the merged model; defaults to <adapter>-merged")
    parser.add_argument("--base-override", default=None,
                        help="use this base instead of the one named in adapter_config.json")
    parser.add_argument("--compare", type=int, default=0,
                        help="generate this many prompts through both models and report "
                             "whether the answers agree; 0 skips the check")
    parser.add_argument("--instruction", default=None,
                        help="instruction appended to each comparison prompt, matching the "
                             "one your training examples used")
    parser.add_argument("--max-new-tokens", type=int, default=96)
    args = parser.parse_args()

    adapter = Path(args.adapter)
    if not (adapter / "adapter_config.json").is_file():
        raise SystemExit(f"{adapter} has no adapter_config.json, so it is not a PEFT adapter "
                         "directory. Point --adapter at what train-lora.py saved.")
    merged = Path(args.merged_dir) if args.merged_dir else adapter.with_name(adapter.name + "-merged")

    recorded_base = PeftConfig.from_pretrained(str(adapter)).base_model_name_or_path
    base_id = args.base_override or recorded_base
    if args.base_override and args.base_override != recorded_base:
        print("WARNING: merging into a base that is not the one recorded in the adapter.")
        print(f"  recorded: {recorded_base}")
        print(f"  using:    {args.base_override}")
        print("  If that is not deliberate, stop now: the result will load and be wrong.")

    print(f"base model: {base_id}")
    print(f"adapter:    {adapter}")
    print(f"merged to:  {merged}")

    # bfloat16 on the CPU: one copy of the weights, no accelerator, and the precision the
    # adapter was trained against. Merging into a 4-bit base applies the update to weights
    # that were already rounded, which is the fault the challenge page reproduces.
    base = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16, device_map="cpu")
    peft_model = PeftModel.from_pretrained(base, str(adapter))

    tokenizer_source = str(adapter) if (adapter / "tokenizer_config.json").is_file() else base_id
    if tokenizer_source == base_id:
        print("WARNING: the adapter directory has no tokeniser, so the base model's is being "
              "used. If training changed the chat template, this export will not match.")
    tokenizer = AutoTokenizer.from_pretrained(tokenizer_source)

    before = []
    prompts = DEFAULT_PROMPTS[: args.compare] if args.compare else []
    for prompt in prompts:
        before.append(generate(peft_model, tokenizer, prompt, args.instruction, args.max_new_tokens))

    merged_model = peft_model.merge_and_unload()   # not in place: the return value is the model
    merged_model.save_pretrained(str(merged))
    tokenizer.save_pretrained(str(merged))
    print(f"merged model written to {merged}")

    if not prompts:
        print("no comparison requested; pass --compare 4 to check the merge before you convert")
        return

    agreed = 0
    for prompt, previous in zip(prompts, before):
        after = generate(merged_model, tokenizer, prompt, args.instruction, args.max_new_tokens)
        same = after.strip() == previous.strip()
        agreed += int(same)
        print(f"\n--- {'MATCH' if same else 'DIFFERS'}: {prompt[:70]}…")
        if not same:
            print(f"  adapter attached: {previous.strip()[:300]}")
            print(f"  merged:           {after.strip()[:300]}")

    print(f"\n{agreed} of {len(prompts)} prompt(s) matched exactly.")
    if agreed != len(prompts):
        print("A merge at the same precision should reproduce the adapter's answers. Differences "
              "here mean the wrong base, the wrong precision or the wrong tokeniser, and they "
              "are worth resolving before you spend an hour converting and quantising.")
    print(json.dumps({"base": base_id, "adapter": str(adapter), "merged": str(merged),
                      "compared": len(prompts), "matched": agreed}))


if __name__ == "__main__":
    main()

Download merge-adapter.py136 lines

RunnableAll tracks

export-gguf.sh

#!/usr/bin/env bash
# Purpose: turn a LoRA adapter into a quantised GGUF file that llama.cpp and everything built
#          on it can serve: merge into the base at bfloat16, convert, quantise, and run one
#          prompt through the result so the export is proved rather than assumed
# Platform: spark, strix, nvidia (and mac for the merge and conversion, although Track M's
#           adapters come from mlx_lm.lora and are fused with mlx_lm.fuse first)
# Minimum memory: 12 GB; the merge needs one bfloat16 copy of the base in system memory
# Assumes: a Python environment with torch, transformers and peft; a llama.cpp checkout
#          holding convert_hf_to_gguf.py; llama-quantize and llama-cli on PATH or pointed at
#          by LLAMA_QUANTIZE and LLAMA_CLI; merge-adapter.py next to this script; and disk
#          for roughly two and a half times the bfloat16 size of the model
#
# Usage: bash export-gguf.sh ADAPTER_DIR [QUANT]
#   ADAPTER_DIR  the directory train-lora.py saved, e.g. runs/format-qwen3-1.7b
#   QUANT        a type llama-quantize accepts; defaults to Q4_K_M
#
# Environment: LLAMA_CPP (default ~/llama.cpp), MERGED_DIR, OUT_DIR, LLAMA_QUANTIZE,
#              LLAMA_CLI, PROMPT, PREDICT, SKIP_RUN=1 to stop before the smoke test.
#              REUSE_MERGED=1 only after checking that the existing merge belongs to
#              this exact adapter and base; otherwise choose a fresh OUT_DIR.
set -euo pipefail

ADAPTER="${1:-}"
QUANT="${2:-Q4_K_M}"
LLAMA_CPP="${LLAMA_CPP:-$HOME/llama.cpp}"
OUT_DIR="${OUT_DIR:-$HOME/models}"
PREDICT="${PREDICT:-96}"
PROMPT="${PROMPT:-The monitoring system reports: the model gateway is unreachable from every machine we have tried this morning.}"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

die() { echo "export-gguf: $*" >&2; exit 1; }

[[ -n "$ADAPTER" ]] || die "usage: bash export-gguf.sh ADAPTER_DIR [QUANT]"
[[ -f "$ADAPTER/adapter_config.json" ]] || die "$ADAPTER has no adapter_config.json; point at what train-lora.py saved"
[[ -f "$LLAMA_CPP/convert_hf_to_gguf.py" ]] || die "convert_hf_to_gguf.py not found under $LLAMA_CPP; set LLAMA_CPP to your llama.cpp checkout"
[[ -f "$HERE/merge-adapter.py" ]] || die "merge-adapter.py is not next to this script"
command -v python3 >/dev/null || die "python3 is not on PATH"

QUANTIZE_BIN="${LLAMA_QUANTIZE:-$(command -v llama-quantize || true)}"
CLI_BIN="${LLAMA_CLI:-$(command -v llama-cli || true)}"
[[ -n "$QUANTIZE_BIN" ]] || die "llama-quantize is not on PATH; set LLAMA_QUANTIZE to its path (built in Part 6)"

NAME="$(basename "$ADAPTER")"
MERGED_DIR="${MERGED_DIR:-$OUT_DIR/$NAME-merged}"
GGUF_BF16="$OUT_DIR/$NAME-bf16.gguf"
GGUF_QUANT="$OUT_DIR/$NAME-$QUANT.gguf"
mkdir -p "$OUT_DIR"

echo "==> 1/4 Merging the adapter into its base at bfloat16, on the CPU"
if [[ -d "$MERGED_DIR" ]]; then
  [[ "${REUSE_MERGED:-0}" == "1" ]] || die "$MERGED_DIR already exists. Choose a fresh OUT_DIR for this run; set REUSE_MERGED=1 only after verifying its adapter/base provenance."
  echo "    Reusing explicitly approved merge: $MERGED_DIR"
else
  python3 "$HERE/merge-adapter.py" --adapter "$ADAPTER" --merged-dir "$MERGED_DIR" --compare 2
fi

echo "==> 2/4 Converting to GGUF at bfloat16"
echo "    Converting at full precision first and quantising afterwards keeps information"
echo "    the quantiser can use. Converting straight to a small type throws it away."
python3 "$LLAMA_CPP/convert_hf_to_gguf.py" "$MERGED_DIR" \
  --outfile "$GGUF_BF16" \
  --outtype bf16

echo "==> 3/4 Quantising to $QUANT"
"$QUANTIZE_BIN" "$GGUF_BF16" "$GGUF_QUANT" "$QUANT"

for f in "$GGUF_BF16" "$GGUF_QUANT"; do
  SIZE=$(wc -c < "$f" | tr -d ' ')
  echo "    $(basename "$f"): $SIZE bytes"
done

if [[ "${SKIP_RUN:-0}" == "1" ]]; then
  echo "==> 4/4 Skipped the smoke test (SKIP_RUN=1)"
else
  [[ -n "$CLI_BIN" ]] || die "llama-cli is not on PATH; set LLAMA_CLI or re-run with SKIP_RUN=1"
  echo "==> 4/4 One prompt through the quantised export, greedily"
  echo "    Compare this with what merge-adapter.py printed for the same prompt. A merge"
  echo "    should match; a quantisation may differ a little. A wholesale change of format"
  echo "    means the chat template did not survive the export."
  "$CLI_BIN" \
    --model "$GGUF_QUANT" \
    --prompt "$PROMPT" \
    --predict "$PREDICT" \
    --temp 0 \
    --seed 0
fi

echo
echo "==> Done."
echo "    merged safetensors: $MERGED_DIR"
echo "    full-precision GGUF: $GGUF_BF16"
echo "    quantised GGUF:      $GGUF_QUANT"
echo "    Delete the full-precision GGUF once the quantised one has been scored, not before."

Download export-gguf.sh93 lines

Track S — NVIDIA DGX Spark

RunnableTrack S · DGX Spark

export the Spark training run

export OUT_DIR="$LAB_DIR/export-format-4b"
bash export-gguf.sh runs/format-qwen3-4b Q4_K_M
export TUNED_GGUF="$OUT_DIR/format-qwen3-4b-Q4_K_M.gguf"
export BASE_REPO="unsloth/Qwen3-4B-GGUF"
export BASE_FILE="Qwen3-4B-Q4_K_M.gguf"

Track X — AMD Ryzen AI Max+ 395

RunnableTrack X · Ryzen AI Max+

export the ROCm training run

export OUT_DIR="$LAB_DIR/export-format-1.7b"
bash export-gguf.sh runs/format-qwen3-1.7b Q4_K_M
export TUNED_GGUF="$OUT_DIR/format-qwen3-1.7b-Q4_K_M.gguf"
export BASE_REPO="unsloth/Qwen3-1.7B-GGUF"
export BASE_FILE="Qwen3-1.7B-Q4_K_M.gguf"

Track M — Apple silicon

The MLX script fused its own adapter. Convert that fused directory, not a PEFT adapter directory.

RunnableTrack M · Apple silicon

export the fused MLX run

export OUT_DIR="$LAB_DIR/export-format-mlx"
mkdir -p "$OUT_DIR"
python3 ~/llama.cpp/convert_hf_to_gguf.py models/format-mlx-fused \
    --outfile "$OUT_DIR/format-bf16.gguf" --outtype bf16
llama-quantize "$OUT_DIR/format-bf16.gguf" "$OUT_DIR/format-Q4_K_M.gguf" Q4_K_M
export TUNED_GGUF="$OUT_DIR/format-Q4_K_M.gguf"
export BASE_REPO="unsloth/Qwen3-1.7B-GGUF"
export BASE_FILE="Qwen3-1.7B-Q4_K_M.gguf"

Record the MLX conversion’s upstream lineage. The community baseline below is a convenient deployment comparison; to isolate the fine-tune strictly, convert the exact unadapted MLX base through the same converter and quantiser as the fused model.

Track N — NVIDIA desktop or laptop

RunnableTrack N · NVIDIA GPU

export the CUDA training run

export OUT_DIR="$LAB_DIR/export-format-1.7b"
bash export-gguf.sh runs/format-qwen3-1.7b Q4_K_M
export TUNED_GGUF="$OUT_DIR/format-qwen3-1.7b-Q4_K_M.gguf"
export BASE_REPO="unsloth/Qwen3-1.7B-GGUF"
export BASE_FILE="Qwen3-1.7B-Q4_K_M.gguf"

Expected result: the selected quantised file exists and the PEFT export’s smoke generation completes. The merge comparison is a diagnostic: inspect task behaviour and numerical differences rather than requiring every sampled string to match. Use a fresh output directory after retraining. The export script refuses implicit reuse of an existing merge so you cannot unknowingly serve an older adapter.

Download the matching base representation if it is not already in your library:

RunnableAll tracks

obtain and check the matching baseline

export BASE_DIR="$HOME/models/$BASE_REPO"
hf download "$BASE_REPO" --include "$BASE_FILE" --local-dir "$BASE_DIR"
export BASE_GGUF="$BASE_DIR/$BASE_FILE"
test -s "$BASE_GGUF"
test -s "$TUNED_GGUF"

Record both files’ revisions and conversion provenance. Equal nominal quantisation is necessary but not sufficient for a strict experiment: community conversions can differ in calibration and source revision. For the controlled result, use the exact base recorded in adapter_config.json and convert it with the same settings as the merge. Label the community-file result separately.

7. Serve and compare two explicit endpoints

Keep Terminal A for the base and Terminal B for the fine-tune. Set the absolute file variables from task 6 again in those shells. Use Terminal C, in LAB_DIR with the evaluation environment active, for the comparison. The names below are deliberate local aliases and do not depend on model size.

RunnableAll tracks

Terminal A: start the base model

: "${BASE_GGUF:?Set BASE_GGUF to the absolute baseline GGUF path from task 6}"
llama-server --model "$BASE_GGUF" --alias format-base --jinja \
    --ctx-size 8192 --host 127.0.0.1 --port 8080

RunnableAll tracks

Terminal B: start the exported fine-tune

: "${TUNED_GGUF:?Set TUNED_GGUF to the absolute exported GGUF path from task 6}"
llama-server --model "$TUNED_GGUF" --alias format-tuned --jinja \
    --ctx-size 8192 --host 127.0.0.1 --port 8081

Wait for readiness, then check both from Terminal C:

RunnableAll tracks

Terminal C: verify model identities

curl --fail --silent --show-error http://127.0.0.1:8080/v1/models
curl --fail --silent --show-error http://127.0.0.1:8081/v1/models

Expected result: the first response lists format-base and the second lists format-tuned. If memory does not permit both processes, start only Terminal A initially and add --pause-before-tuned to the comparison below. At its prompt, stop A with Ctrl+C, start B, verify B’s model listing and press Enter. Before the next task file the harness waits again so you can stop B and restore A. This is a supported sequential procedure, not a race to swap models while a script continues running.

RunnableAll tracks

evaluate-against-base.py

"""Score a fine-tuned model and the base model it came from on the same tasks, and report the gap.

Purpose: the only question this part asks. Runs Part 10's evaluation harness twice over the
         same task files at the same settings, once against the base model and once against
         the fine-tune, optionally grades both with the same judge, and prints the difference
         per category so that a gain on the task you trained for and a loss on everything else
         are both visible. Appends one record to the lab notebook.
Platform: all (pure Python over HTTP; both models are reached through an OpenAI-compatible API,
          so they may be served by any engine on any track, or by the Part 9 gateway)
Minimum memory: 8 GB on the machine serving the models; this script needs very little
Assumes: Python 3.10 or newer; Part 10's run-eval.py and judge.py in --harness-dir; both models
         reachable at --base-url under the names given, one at a time or together; sftlog.py
         next to this file. Pass every task file you care about: the one that tests the
         behaviour you trained, and your own Part 10 set, which is where a regression shows up.

Usage: python3 evaluate-against-base.py --harness-dir ~/eval \
           --base-url http://127.0.0.1:8080/v1 \
           --base-model qwen3-1.7b --tuned-model qwen3-1.7b-format \
           --quant Q4_K_M --engine llama.cpp --engine-version v0.4.0 \
           --tasks format-tasks.json --tasks my-tasks.json \
           --out-dir eval-out --labbook labbook.md

       Add --judge-model qwen3-8b to grade both runs with the same judge. Leave it out and
       only the deterministic checks are compared, which is a floor rather than a score but
       is exactly repeatable and needs no third model.
"""
from __future__ import annotations

import argparse
import json
import os
import subprocess
import sys
import shlex
from collections import defaultdict
from pathlib import Path
from typing import Any

import sftlog


def run_harness(script: Path, arguments: list[str]) -> None:
    """Run one harness script, showing the command first so the log says what happened."""
    command = [sys.executable, str(script), *arguments]
    safe_command = list(command)
    for i, value in enumerate(safe_command[:-1]):
        if value == "--api-key":
            safe_command[i + 1] = "[redacted]"
    print("+ " + shlex.join(safe_command))
    subprocess.run(command, check=True)


def category_pass_rates(results: list[dict[str, Any]]) -> dict[str, tuple[int, int]]:
    """Deterministic passes and totals per category. No model involved, exactly repeatable."""
    counts: dict[str, list[int]] = defaultdict(lambda: [0, 0])
    for item in results:
        counts[item["category"]][1] += 1
        counts[item["category"]][0] += int(item["checks"]["passed"])
    return {k: (v[0], v[1]) for k, v in sorted(counts.items())}


def evaluate(args, script_dir: Path, out_dir: Path, tasks_path: Path,
             model: str, label: str) -> dict[str, Any]:
    """One model, one task file: run it, optionally grade it, and return the numbers."""
    stem = f"{label}-{tasks_path.stem}"
    results_path = out_dir / f"results-{stem}.json"
    arguments = [
        "--base-url", (args.tuned_url or args.base_url) if label == "tuned" else args.base_url,
        "--model", model,
        "--quant", args.quant,
        "--engine", args.engine,
        "--engine-version", args.engine_version,
        "--tasks", str(tasks_path),
        "--out", str(results_path),
        "--notes", f"part-13 {label} on {tasks_path.name}",
    ]
    if args.api_key:
        arguments += ["--api-key", args.api_key]
    if args.system:
        arguments += ["--system", args.system]
    if args.labbook:
        arguments += ["--labbook", args.labbook]
    run_harness(script_dir / "run-eval.py", arguments)

    payload = json.loads(results_path.read_text(encoding="utf-8"))
    out: dict[str, Any] = {
        "model": model,
        "results_file": str(results_path),
        "tasks": payload["run"]["tasks"],
        "checks_passed": payload["run"]["checks_passed"],
        "by_category_checks": category_pass_rates(payload["results"]),
        "judge_mean": None,
        "by_category_judge": {},
    }

    if args.judge_model:
        judged_path = out_dir / f"judged-{stem}.json"
        judge_arguments = [
            "grade",
            "--base-url", args.judge_url or args.base_url,
            "--judge-model", args.judge_model,
            "--results", str(results_path),
            "--out", str(judged_path),
        ]
        if args.api_key:
            judge_arguments += ["--api-key", args.api_key]
        if args.labbook:
            judge_arguments += ["--labbook", args.labbook]
        run_harness(script_dir / "judge.py", judge_arguments)
        judged = json.loads(judged_path.read_text(encoding="utf-8"))
        out["judge_mean"] = judged["judge"]["judge_mean"]
        out["by_category_judge"] = judged["judge"]["by_category"]
        out["judged_file"] = str(judged_path)
    return out


def delta(new: float | None, old: float | None) -> str:
    if new is None or old is None:
        return "—"
    difference = new - old
    return f"{difference:+.2f}"


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--harness-dir", default=".",
                        help="directory holding Part 10's run-eval.py and judge.py")
    parser.add_argument("--base-url", default="http://127.0.0.1:8080/v1")
    parser.add_argument("--tuned-url", default=None,
                        help="fine-tune endpoint; defaults to --base-url for a shared gateway")
    parser.add_argument("--pause-before-tuned", action="store_true",
                        help="wait for you to stop the base server and start the fine-tune")
    parser.add_argument("--api-key", default=os.environ.get("OPENAI_API_KEY"))
    parser.add_argument("--base-model", required=True,
                        help="the name the endpoint answers to for the base model")
    parser.add_argument("--tuned-model", required=True,
                        help="the name the endpoint answers to for the fine-tune")
    parser.add_argument("--quant", required=True,
                        help="the quantisation both models are serving at; a comparison "
                             "across two quantisations measures the quantisation as well")
    parser.add_argument("--engine", default="llama.cpp")
    parser.add_argument("--engine-version", default="unknown")
    parser.add_argument("--system", default=None,
                        help="system prompt sent with every task; use the one you trained with")
    parser.add_argument("--tasks", action="append", required=True,
                        help="a task file; repeat the flag for more than one")
    parser.add_argument("--judge-model", default=None,
                        help="grade both runs with this model; leave out for checks only")
    parser.add_argument("--judge-url", default=None,
                        help="endpoint for the judge, if it is not at --base-url")
    parser.add_argument("--out-dir", default="eval-out")
    parser.add_argument("--labbook", default=None)
    parser.add_argument("--notes", default=None)
    args = parser.parse_args()

    script_dir = Path(args.harness_dir).expanduser()
    for name in ("run-eval.py", "judge.py") if args.judge_model else ("run-eval.py",):
        if not (script_dir / name).is_file():
            raise SystemExit(f"{script_dir / name} not found; --harness-dir must point at the "
                             "directory holding Part 10's harness scripts")
    if args.judge_model in (args.base_model, args.tuned_model):
        print("WARNING: the judge is one of the models under test. Part 10's lab covers "
              "self-preference bias; use a third, larger model if you can.")

    out_dir = Path(args.out_dir)
    out_dir.mkdir(parents=True, exist_ok=True)

    comparison: dict[str, Any] = {}
    for tasks in args.tasks:
        tasks_path = Path(tasks)
        if not tasks_path.is_file():
            raise SystemExit(f"{tasks_path} does not exist")
        print(f"\n=== {tasks_path.name}: base model {args.base_model}")
        base = evaluate(args, script_dir, out_dir, tasks_path, args.base_model, "base")
        if args.pause_before_tuned:
            input("Base results saved. Stop the base server, start the fine-tune, "
                  "verify its readiness, then press Enter: ")
        print(f"\n=== {tasks_path.name}: fine-tune {args.tuned_model}")
        tuned = evaluate(args, script_dir, out_dir, tasks_path, args.tuned_model, "tuned")
        comparison[tasks_path.name] = {"base": base, "tuned": tuned}
        if args.pause_before_tuned and tasks != args.tasks[-1]:
            input("Fine-tune results saved. Restore the base server for the next task file, "
                  "verify readiness, then press Enter: ")

    print("\n" + "=" * 78)
    print("BASE VERSUS FINE-TUNE, same tasks, same settings, same quantisation")
    print("=" * 78)
    for name, pair in comparison.items():
        base, tuned = pair["base"], pair["tuned"]
        print(f"\n{name}   ({base['tasks']} task(s))")
        print(f"  deterministic checks   base {base['checks_passed']:>3}/{base['tasks']}"
              f"   fine-tune {tuned['checks_passed']:>3}/{tuned['tasks']}"
              f"   change {tuned['checks_passed'] - base['checks_passed']:+d}")
        if base["judge_mean"] is not None:
            print(f"  judge mean             base {base['judge_mean']:>5}"
                  f"   fine-tune {tuned['judge_mean']:>5}"
                  f"   change {delta(tuned['judge_mean'], base['judge_mean'])}")
        categories = sorted(set(base["by_category_checks"]) | set(tuned["by_category_checks"]))
        for category in categories:
            b_pass, b_total = base["by_category_checks"].get(category, (0, 0))
            t_pass, t_total = tuned["by_category_checks"].get(category, (0, 0))
            line = (f"    {category:14s} checks {b_pass}/{b_total} -> {t_pass}/{t_total}"
                    f"  ({t_pass - b_pass:+d})")
            if base["by_category_judge"] or tuned["by_category_judge"]:
                b_judge = base["by_category_judge"].get(category)
                t_judge = tuned["by_category_judge"].get(category)
                line += f"   judge {b_judge} -> {t_judge} ({delta(t_judge, b_judge)})"
            print(line)

    print("\nRead the categories, not the total. A fine-tune that gains on the behaviour you")
    print("trained and loses on two others can leave the overall mean unchanged, and that")
    print("trade is the finding. Anything smaller than the difference between two runs of")
    print("the base model at these settings is not a difference.")

    if args.labbook:
        record = sftlog.record(
            labbook=args.labbook,
            lab="part-13/evaluate-against-base",
            model=f"{args.tuned_model} vs {args.base_model}",
            dataset={"task_files": args.tasks,
                     "sha256": {t: sftlog.file_sha256(t) for t in args.tasks}},
            hyperparameters={"base_url": args.base_url,
                             "tuned_url": args.tuned_url or args.base_url, "quant": args.quant,
                             "engine": args.engine, "engine_version": args.engine_version,
                             "judge_model": args.judge_model, "system_prompt": args.system},
            seed=0,
            losses={},
            scores={name: {
                "base_checks_passed": pair["base"]["checks_passed"],
                "tuned_checks_passed": pair["tuned"]["checks_passed"],
                "tasks": pair["base"]["tasks"],
                "base_judge_mean": pair["base"]["judge_mean"],
                "tuned_judge_mean": pair["tuned"]["judge_mean"],
                "base_by_category_judge": pair["base"]["by_category_judge"],
                "tuned_by_category_judge": pair["tuned"]["by_category_judge"],
            } for name, pair in comparison.items()},
            config_path=__file__,
            notes=args.notes,
        )
        print(f"\nrecorded comparison {record['run_id']} in {args.labbook}")


if __name__ == "__main__":
    main()

Download evaluate-against-base.py244 lines

Copy your frozen Part 10 evaluation set into this directory if it is not already here:

RunnableAll tracks

check the two evaluation inputs

cp "$LABS_ROOT/part-10-models-at-work/my-tasks.json" ./my-tasks.json
python3 -m json.tool format-tasks.json > /dev/null
python3 -m json.tool my-tasks.json > /dev/null

If the copy fails, complete Part 10’s task-writing step or use the actual path where you saved it. Do not replace the personal regression set with training examples just to make the command run.

RunnableAll tracks

compare the same task files through both endpoints

python3 evaluate-against-base.py \
    --harness-dir "$LABS_ROOT/part-10-models-at-work" \
    --base-url http://127.0.0.1:8080/v1 \
    --tuned-url http://127.0.0.1:8081/v1 \
    --base-model format-base --tuned-model format-tuned \
    --quant Q4_K_M --engine llama.cpp --engine-version v0.4.0 \
    --tasks format-tasks.json --tasks my-tasks.json \
    --out-dir eval-out --labbook labbook.md

Replace the recorded engine version with your actual build. Expected result: four result files, base and tuned for each task set, followed by per-category pass counts. Inspect failed outputs; required-string checks do not fully establish semantic correctness. A fine-tune that fails to improve is a valid experimental outcome and should lead to diagnosis rather than a fabricated win.

8. Promote only the evaluated artefact

Keep the base alias and add a new gateway alias for the exact TUNED_GGUF you measured. In the Part 9 gateway configuration, use its absolute path and place the alias in the same memory-exclusion group as other large models. The following is an edit fragment: replace the model path before use.

Fragment — not complete on its own

  local/triage:
    name: Evaluated ticket-triage fine-tune
    cmd: |
      ${server}
      --model /absolute/path/to/the/evaluated-fine-tune.gguf
      --alias local/triage
      --ctx-size 8192
      --jinja
    checkEndpoint: /health
    ttl: 900

Stop the comparison servers before restarting the gateway if they compete for memory or ports. Repeat a format task through local/triage and retain the response. Record the alias-to-file mapping, base lineage, dataset checksum and evaluation directory with your deployment configuration. If the model fails the task contract, leave the base active and retain the failed candidate for diagnosis.

Check the experiment before accepting the fine-tune

Use a short acceptance table with one row per stage:

Stage	Required observation	If it fails
Data	Targets follow the written format, including edge cases	Repair labels and regenerate into a new dataset revision
Split	Personal evaluation tasks remain outside training	Resolve overlap before reporting any improvement
Training	Intended adapters receive updates and losses remain finite	Inspect targets, masks, dtype and module selection
Export	The exported file loads and retains the target behaviour	Compare adapter, merge and conversion separately
Comparison	Base and tuned aliases are reached at the intended URLs	Repair endpoint mapping before interpreting scores
Promotion	The gateway serves the exact file evaluated	Correct the alias mapping and repeat the task probe

Keep the two evaluation sets separate in the conclusion. The format set measures the behaviour you trained; my-tasks.json measures broader regressions. Required strings are useful checks but do not prove that severity and action labels are semantically right, so read failed and borderline answers. If training produces no improvement, retain the result and investigate the challenge lesson. The experiment can be completed honestly without promoting the candidate.

Validation

You are done when all of the following are true:

• data/train.jsonl, data/valid.jsonl and format-tasks.json exist, and you have read ten training examples including two edge cases;
• decontaminate.py reports no exact matches between your training data and my-tasks.json, or you have read and resolved the ones it found;
• the training run printed a trainable-parameter percentage consistent with the adapter arithmetic, and finished with a best_epoch you can explain;
• labbook.md contains the training run record, with the dataset checksum, every hyperparameter and the losses;
• evaluate-against-base.py has produced a comparison over both task files, and it is in labbook.md;
• the format comparison is recorded, including a failed improvement if that is what you observed;
• you can say what happened to your own Part 10 task set, whether it went up, down or nowhere;
• a converted and quantised file exists, and its comparison was inspected for regressions;
• the fine-tune answers under its own name through the gateway.

Expected outcome

A model that holds your format, and a record that says by how much.

Pending validationBase versus fine-tune, your recording sheet

Task set	Model	Checks passed	Judge mean	Change
format-tasks.json	base	—	—	—
format-tasks.json	fine-tune	—	—	—
my-tasks.json	base	—	—	—
my-tasks.json	fine-tune	—	—	—

your machine: track, chip and memory, your operating system and version · llama.cpp, or mlx-lm on Track M the build number from llama-cli --version · the base model you chose, and the adapter run id from labbook.md, the same quantisation for both rows of each pair · 8,192 tokens of context · the date you ran it

Empty on purpose. Four rows, two task sets, one variable between the rows of each pair. Fill it from evaluate-against-base.py's summary. A fifth and sixth row are worth adding if you run the base model twice: the difference between two runs of the same model is your noise floor, and any change smaller than it is not a change.

Pending validationTraining run, your recording sheet

Track	Base model	Rank	Epochs	Best epoch	Best eval loss	Wall-clock
S	—	—	—	—	—	—
X	—	—	—	—	—	—
M	—	—	—	—	—	—
N	—	—	—	—	—	—

one row per machine you run this on · PyTorch with Transformers, TRL and PEFT; mlx-lm on Track M the package versions sftlog.py recorded in labbook.md · as listed, BF16 base, no quantisation during training · 1,024 tokens of context · the date you ran it

Empty on purpose. The validation pass fills these in per track with measured wall-clock and peak memory. The loss values are only comparable down a column for the same dataset and sequence length: two runs on different data produce losses that are not on the same scale.

The finding that surprises people is usually one of two. Either the format gain is very large and the rest is unchanged, which is what a well-scoped format fine-tune looks like and is more satisfying than it sounds. Or the format gain is large and something in your own task set went down, which is the trade the first lesson warned about, arriving on schedule and in a form you can measure.

Troubleshooting

The trainable-parameter percentage is far below what the arithmetic predicted. The target module selection may have matched fewer layers than intended, or the rank differs from your estimate. PEFT normally rejects a selection that matches no modules; a low count alone does not prove that no adapter was attached. Re-run with --list-modules, or use --target-modules all-linear.

The script says the model has no chat template. You picked a base checkpoint rather than an instruct one. Use the instruct model, or set a template deliberately with chat_template_path, which is the Part 12 path rather than this one.

Out of memory partway through, on a run that started fine. Almost never a leak. It is the longest example in the dataset arriving in a batch, because activation memory scales with the actual sequence length being processed. Lower --max-length, or --batch-size with a matching rise in --grad-accum, or add --gradient-checkpointing.

The loss is nan after a few steps. The learning rate is too high for this configuration, or the run is in float16 somewhere. Halve --lr and check the precision line the script printed.

The device line says cpu on a machine with an accelerator. Part 11’s environment lesson. On Track X in particular, a PyTorch wheel without ROCm installs cleanly and trains on the CPU without complaining.

The fine-tune scores worse than the base on the format tasks. That is this part’s challenge page, and it is a genuinely useful outcome. Collect the evidence before theorising: the chat template used in training against the one used in serving, the sampling settings on both sides, the overlap between the two files, and which base the adapter was merged into.

evaluate-against-base.py cannot find the harness. --harness-dir must point at the directory holding Part 10’s run-eval.py and judge.py, which the cleanup step of that lab suggested putting in ~/eval.

The merged model’s answers differ from the adapter’s. Read the warnings the merge printed. The two usual causes are a tokeniser that came from the base rather than from the adapter directory, and a --base-override that pointed at a different checkpoint.

Cleanup

Keep the adapter, dataset, task files, export provenance and notebook. Preserve the evaluated quantised file at the gateway path; only remove intermediates after verifying your archive.

RunnableAll tracks

archive the experiment before optional cleanup

mkdir -p "$HOME/finetunes/format-experiment"
cp -R runs data format-tasks.json my-tasks.json eval-out labbook.md "$HOME/finetunes/format-experiment/"

Inspect the archive and retain the exact base and exported model paths. No deletion is required to finish this lab; remove large intermediates manually only when you have identified which files can be reproduced and which the gateway still serves.

The training process leaves nothing running. If you started servers for the comparison, stop them.

What you learned

• A format fine-tune is the cheapest real fine-tune. A few hundred consistent examples, minutes of training on a small model, and a change you can measure with string checks that need no judge.
• The dataset decides the result and the checks decide whether you can see it. The generator makes labels correct by construction; the decontamination check makes the regression number mean something.
• Two task sets, not one. The set you trained for shows the gain. Your own set from Part 10 shows the cost, and a fine-tune reported without it is half a result.
• The trainable-parameter count is a check, not a decoration. It is the fastest way to catch an unintended target-module selection, and it costs one line.
• The best epoch is a message about your data. Best epoch one means too many epochs for this much data; a best epoch in the middle means the run and the dataset agree.
• Serving an adapter is quicker than merging one. Convert the adapter, attach it with --lora, score it, and only merge when you have decided to keep it.
• A fine-tune with no name in the gateway is a fine-tune you will not use. One block in llama-swap.yaml, next to the base model it came from.

Record in the notebook: your track and machine; the format contract you chose; the dataset size, seed and checksum; the base model, rank, alpha, target modules, learning rate, epochs and effective batch; the best epoch and the best evaluation loss; wall-clock for the run; the four rows of the comparison sheet with the quantisation and settings; what happened to each category of your own task set; and one sentence naming the thing that surprised you.

Check your understanding

Question 1. Why does the lab score the fine-tune on two task files rather than one?

• To double the sample size
• The generated format set measures the gain on the behaviour you trained; your own Part 10 set measures whether anything else got worse
• Because the harness requires two files
• To compare two judges

Show the answer and why

Answer: The generated format set measures the gain on the behaviour you trained; your own Part 10 set measures whether anything else got worse

A fine-tune that gains on one behaviour and loses on three can leave an overall mean flat. Only a set written before this fine-tune existed can show the loss, which is why the Part 10 file is the one that matters most here.

Question 2. The training script prints a trainable-parameter percentage far below what the adapter arithmetic predicts. What happened?

• The learning rate is too low
• The names passed to --target-modules did not match this architecture, so fewer layers were adapted than intended
• The dataset is too small
• Gradient checkpointing is on

Show the answer and why

Answer: The names passed to --target-modules did not match this architecture, so fewer layers were adapted than intended

An entirely unmatched selection raises an error; a partial match can still produce an unintended adapter layout. Run with --list-modules to see what the model actually has, or pass --target-modules all-linear.

Question 3. You score a BF16 fine-tune against a Q4_K_M base and the fine-tune wins. What have you measured?

• The effect of fine-tuning
• The fine-tune and the precision difference together, since two variables moved between the runs
• The judge's bias
• Nothing at all

Show the answer and why

Answer: The fine-tune and the precision difference together, since two variables moved between the runs

One variable per comparison. Serve both at the same quantisation with the same sampling settings and the same system prompt, or the number cannot be attributed to the thing you changed.

Question 4. Why does the comparison use separate base and tuned server addresses?

• Different ports make the models more accurate
• Each request must reach the intended artifact; model names alone do not switch a single-model server
• Both models must remain resident for the entire experiment
• Quantisation requires a different API protocol

Show the answer and why

Answer: Each request must reach the intended artifact; model names alone do not switch a single-model server

The harness sends base requests to --base-url and tuned requests to --tuned-url. Check the loaded artifacts and aliases at both endpoints. If memory only permits one server, use --pause-before-tuned and switch the loaded model at each checkpoint.

Question 5. Which of these are genuine reasons the run might finish with best_epoch equal to 1? Select all that apply.

• Three epochs is too many for a few hundred examples
• The learning rate is high enough that the model overfits within one pass
• The validation split leaked into the training split, so the evaluation loss was optimistic from the start
• Gradient accumulation was set too high

Show the answer and why

Answer: Three epochs is too many for a few hundred examples, The learning rate is high enough that the model overfits within one pass, The validation split leaked into the training split, so the evaluation loss was optimistic from the start

The first two are the usual causes and the script prints a note about them. The third is subtler and shows up as an evaluation loss that tracks the training loss too closely. Gradient accumulation changes the effective batch, not whether the model overfits at epoch one.

Check my answersClear