# Distillation report: <your student name>

<!--
Purpose: the written deliverable of Part 15's project. One page that lets somebody else, or
         you in six months, decide whether this student is worth serving, whether the result
         can be believed, and what it would cost to do again on a different task.
Platform: all
Minimum memory: not applicable; this is a document
Assumes: the stage records that labbook.md accumulated, the filter report, the rejection
         report if you took the reasoning route, and the table evaluate-triplet.py printed.
         Fill in every angle-bracket placeholder and delete every comment block, including
         this one. A field you cannot fill is a field to write "not recorded" in, not one to
         delete: the gap is information, and it is usually the most useful information here.

Keep it to one page plus the tables. Every number carries its context, which is the course's
rule about numbers applied to your own work.
-->

**Author:** &lt;you&gt; &nbsp; **Date:** &lt;YYYY-MM-DD&gt; &nbsp; **Track:** &lt;S, X, M or N&gt;
**Machine:** &lt;chip and memory, from the hardware reference&gt;
**Configuration hash:** &lt;the config_sha256 pipeline.py printed&gt;

## What this student is for

<!-- Two or three sentences. The task, the machine it has to run on, and why a smaller model
     was wanted at all. "It fits the laptop" is a legitimate reason and should be stated. -->

## The claim

<!-- One sentence with a number in it, written before the run and unchanged afterwards. The
     shape: "On <task set>, at temperature <t> with seed <s>, the distilled <student> closes
     <n>% of the gap between <base student> and <teacher> in the deterministic checks, and is
     not worse in any category." -->

## The pair

| Field | Teacher | Student |
| --- | --- | --- |
| Model id | &lt;course id&gt; | &lt;course id&gt; |
| Repository | &lt;from the model card&gt; | &lt;from the model card&gt; |
| Parameters | &lt;total / active&gt; | &lt;total&gt; |
| Licence | &lt;from the model card&gt; | &lt;from the model card&gt; |
| Terms on using outputs to train | &lt;quote the clause, or "the licence says nothing"&gt; | not applicable |
| Quantisation served at | &lt;e.g. Q4_K_M&gt; | &lt;BF16 for training, a quant for serving&gt; |
| Same tokeniser? | &lt;yes or no, and how you checked&gt; | |

<!-- The licence row is not decoration. A student trained on another model's outputs inherits
     that model's terms wherever the terms say so, and the naming conditions of some licences
     apply to the model you produce, not only to the one you used. -->

## Route and settings

| Field | Value |
| --- | --- |
| Route | &lt;sequence-level, logit, or reasoning traces with rejection sampling&gt; |
| Seed prompts | &lt;count, and how they were seeded&gt; |
| Generation settings | &lt;temperature, top-p, samples per prompt, thinking mode&gt; |
| Raw teacher answers | &lt;count&gt; |
| Kept after filtering | &lt;count, and the percentage&gt; |
| Top three rejection reasons | &lt;from filter-report.json&gt; |
| Decontaminated against | &lt;task file, and the containment threshold&gt; |
| Trainer and settings | &lt;SFTTrainer, DistillationTrainer or GKDTrainer; rank, epochs, learning rate&gt; |
| Versions | &lt;transformers, trl, peft, or mlx-lm&gt; |
| Seeds | &lt;the seed every stage used&gt; |

## Result

<!-- The table evaluate-triplet.py wrote to benchmark-rows.json. Three rows, one task set, one
     set of sampling settings, and the settings named above the table. Do not report a judge
     mean without saying which model judged and whether you measured its agreement with you. -->

| Model | Served as | Deterministic checks | Judge mean, 1-5 |
| --- | --- | --- | --- |
| Teacher | &lt;alias&gt; | &lt;n of total&gt; | &lt;mean, or "not judged"&gt; |
| Base student | &lt;alias&gt; | &lt;n of total&gt; | &lt;mean, or "not judged"&gt; |
| Distilled student | &lt;alias&gt; | &lt;n of total&gt; | &lt;mean, or "not judged"&gt; |

**Gap closed:** &lt;percentage, from evaluate-triplet.py&gt;

**Categories where the distilled student fell below the base:** &lt;list them, or "none"&gt;

<!-- A category that got worse is the finding, whatever the total says. Part 15's challenge is
     about exactly this, and a report that hides it is a report that will mislead you later. -->

## Cost

| Stage | Completion tokens | Wall clock | Energy |
| --- | --- | --- | --- |
| Generate | &lt;from the run record&gt; | &lt;minutes&gt; | &lt;Wh, or "not reported here"&gt; |
| Filter | not applicable | &lt;minutes&gt; | negligible |
| Train | not applicable | &lt;minutes&gt; | &lt;Wh, or "not reported"&gt; |
| Evaluate | &lt;from the run record&gt; | &lt;minutes&gt; | &lt;Wh, or "not reported"&gt; |
| **Total** | | | |

<!-- Energy is null on Track M, where an unprivileged process is not told the figure. Write
     "not reported on this machine" rather than zero: a missing measurement is not a small one. -->

## What did not work

<!-- The honest section. Thresholds you had to change and why; prompts the teacher answered
     badly; a category the student never learned; a stage you had to run twice. A report with
     nothing here is a report whose author stopped reading their own output. -->

## Would you do it again this way?

<!-- Two or three sentences. Compare against the alternatives you did not take: a bigger
     quantised model served directly, a fine-tune on human data, prompting the base student
     harder, or retrieval. Name the one you would try next and what would make you change
     your mind. -->

## Reproducing this

```
python3 pipeline.py --config <your config file> --dry-run
python3 pipeline.py --config <your config file>
```

| Field | Value |
| --- | --- |
| Configuration file | &lt;name, and where it is kept&gt; |
| Configuration hash | &lt;config_sha256&gt; |
| Seed prompt file SHA-256 | &lt;from make-seed-prompts.py&gt; |
| Training file SHA-256 | &lt;from the filter run record&gt; |
| Lab notebook lines | &lt;the run ids of every stage&gt; |