# Model improvement — <the behaviour you are trying to change>

<!--
Purpose: the deliverable of Capstone 4. One target behaviour, one evaluation set built
         before any training, one technique chosen with a reason, the run logs, a
         before-and-after comparison with its variance, the quantised export measured as
         the separate model it is, and the regressions stated rather than buried.
Platform: all; the technique is chosen by what your memory tier can train
Minimum memory: 12 GB for the smallest path in the course; more for the larger techniques
Assumes: the training environment from Part 11, the fine-tuning workflow of Part 13, the
         preference and reinforcement-learning labs of Part 14 or the distillation
         pipeline of Part 15, and the evaluation methods of Part 16. Run records come from
         Part 11's runlog and evaluation records from Part 16's scripts.

The rule this template exists to enforce: the evaluation set is written down, dated and
counted before the first training run starts. Everything else here is bookkeeping around
that one discipline. Delete every comment block, including this one.
-->

**Author:** <you> · **Date:** <YYYY-MM-DD> · **Track:** <S/X/M/N> ·
**Base model:** <exact repository id> · **Base model licence:** <from the model card>

---

## 1. The target behaviour

**What the model should do differently, in one sentence:** <specific enough to be wrong.
"Answers in our house JSON shape without a preamble" is specific; "is better at coding" is
not.>

**Why a smaller model with this behaviour is worth more to you than a larger one without
it:** <one paragraph. If the honest answer is that it is not, that is a finding and the
project can report it.>

**The claim, written before training and unchanged afterwards:** <one sentence with a
comparison in it: "on <task file>, at temperature <t> with seeds <s>, the improved model
passes more of the deterministic checks than the base model and its judge mean in <these
categories> is not lower.">

## 2. The evaluation set

| Field | Value |
| --- | --- |
| File and format | <name; one task per line> |
| Items | <n> |
| Built on | <YYYY-MM-DD; before the first run record, and here is its date> |
| How items were chosen | <from real use, from your own work, written by hand, generated and reviewed> |
| Deterministic checks | <how many items are checkable without a judge> |
| Judged items | <how many, by which judge model, position-swapped> |
| Categories | <the groups you will report separately> |
| Held out from training | <how you kept them apart> |
| Decontamination | <what you checked the training data against, what overlap was found, what you did> |
| Licence and confidentiality of the material | <what the source allows> |

**The noise floor:** <evaluate the base model twice, unchanged, and record the spread
here.> Any later difference smaller than <the spread> is reported as no difference.

<This row is the one that turns the rest of the document into a measurement. Part 16's
quantisation lab measures one file twice for exactly this reason, and finding that two
things are indistinguishable is an honest and useful result.>

## 3. The technique, and why

| Option | Fits my memory tier? | Would it change the behaviour in section 1? | Chosen? |
| --- | --- | --- | --- |
| Supervised fine-tuning with LoRA or QLoRA | | | |
| Preference optimisation on top of the fine-tune | | | |
| Reinforcement learning with a verifiable reward | | | |
| Distillation from a larger local teacher | | | |

**Chosen:** <technique, or the sequence: fine-tune then preference optimisation.>
**Because:** <two sentences. Memory is a reason; so is "the behaviour is a format, and
imitation is the cheapest way to teach a format".>

**Memory arithmetic for the run, from Part 11:** weights <GB> + gradients <GB> +
optimiser states <GB> + activations <GB> = <GB> against <GB> available. **Peak memory
actually observed:** <GB>. **Gap, and why:** <one sentence.>

## 4. The data

| Field | Value |
| --- | --- |
| Source | <your own work, an agent's trajectories, a teacher's generations, a public dataset> |
| Licence | <exact> |
| Training examples | <n> |
| Validation examples | <n> |
| Kept after filtering | <n, and what the filter rejected> |
| Redaction | <what was removed before the trainer saw it, or "none needed"> |
| Training file hash | <from the run record> |

## 5. The runs

| Run id | Technique | Key settings | Seed | Final loss or reward | Attended minutes | Total minutes | Outcome |
| --- | --- | --- | --- | --- | --- | --- | --- |
| <from labbook.md> | | <rank, alpha, learning rate, epochs, batch, beta> | | | | | <kept / discarded, and why> |

**Runs that were discarded:** <how many, and what went wrong. A project with no discarded
runs either got lucky or is not telling you everything.>

**What the curves said:** <loss, reward, length and any divergence measure, read together.
A reward that rose while the answers got longer and worse is the classic finding and it is
worth reporting when it happens.>

## 6. Before and after

<Same task file, same shots, same chat template, same sampling settings, same seeds, both
models. Three runs of each at minimum, so that a difference can be told from a wobble.>

| Model | Quantisation | Metric | Runs | Mean | Range | Compared with |
| --- | --- | --- | --- | --- | --- | --- |
| Base | <as served> | <deterministic checks passed> | <3> | | | reference |
| Improved | <as served> | <deterministic checks passed> | <3> | | | base |
| Base | | <judge mean, position-swapped> | | | | reference |
| Improved | | <judge mean, position-swapped> | | | | base |

| Category | Base mean | Improved mean | Range of each | Better, worse or indistinguishable |
| --- | --- | --- | --- | --- |
| <category> | | | | |

**Did the claim in section 1 hold?** <yes, no, or partly, in one sentence, against the
noise floor from section 2.>

**On a standard suite:** <the tasks, the harness and its version, the shot count, the
chat-template setting, the number of items scored, the runs and the spread. Or "not run",
with the reason.>

## 7. Regressions and limits

| What got worse | By how much | Against what noise floor | Acceptable for my use? | Why |
| --- | --- | --- | --- | --- |
| <category or behaviour> | | | <yes/no> | |

**What the model still cannot do:** <the things the target behaviour did not touch, stated
plainly, so that nobody deploys it expecting them.>

**What would break it:** <inputs outside the training distribution, a different chat
template, a longer context than you trained for.>

## 8. Quantisation and export

| Field | Value |
| --- | --- |
| Merged or adapter kept separate | <which, and why> |
| Export format | <GGUF, MLX, safetensors> |
| Quantisation | <exact type> |
| File size | <GB> |
| Damage measured how | <divergence against the unquantised model, and your task set> |
| Task set result, quantised | <mean and range over the same number of runs> |
| Difference from the unquantised model | <against the noise floor> |
| Served under which gateway name | <alias> |

<The quantised file is a different model from the adapter you trained, and Part 16's rule
is to measure the file you will actually serve. A report that evaluates the unquantised
model and serves the quantised one has measured something nobody uses.>

## 9. What this does not establish

<What you did not measure, what one run cannot distinguish, which comparisons were not
strictly like for like, and what you took from documentation without verifying. Three
items is a minimum, and "I ran each configuration once" belongs here whenever it is true.>