# Capstone rubric — the checks you apply to your own work

<!--
Purpose: the self-assessment for Part 28. This course has no marker, so the rubric is
         written as questions you can answer yes or no from two files you already have:
         the lab notebook you kept since Part 1, and the report you write in Capstone 6.
         A question you cannot answer from those two files is a "no", however sure you
         feel. That rule is the whole rubric.
Platform: all
Minimum memory: not applicable; this is a document
Assumes: labbook.md from Part 1 onwards, and the six capstone deliverables in whatever
         state they have reached. Work through it twice: once when you think you are
         finished, and once a week later, when you are reading your own report as a
         stranger would.

How to use it: answer every question in the column that says what would satisfy it.
Count the "no" answers rather than scoring yourself out of anything. Each "no" is one
sentence to add to the report's "what is not true here" section, and that section is
what makes the rest of the document worth reading.
-->

**Reader:** <you> · **Date of this pass:** <YYYY-MM-DD> · **Pass:** <first / second>

---

## A. The four questions that apply to every deliverable

These are the course's standard, applied to your own work. If a deliverable fails one of
these, fixing it is worth more than adding anything new.

| # | Question | What a "yes" looks like | Where to look |
| --- | --- | --- | --- |
| A1 | Does every number carry its context? | Each figure sits in a table naming the machine, the operating system, the engine and its version, the model, the quantisation, the context length and the date. No figure appears in a sentence on its own. | The report's tables; the records in `labbook.md` |
| A2 | Does every claim point at a measurement? | Every sentence asserting that something is faster, better, smaller or sufficient names a labelled measurement that exists in the notebook. Sentences that cannot are marked as arithmetic or as a vendor figure, with the source. | The report's prose, sentence by sentence |
| A3 | Does the single-machine variant say what a second machine would change? | A named machine, the role it would take, the link it would need, and the measurement that motivates it. "It would be faster" is a "no". | Capstone 3, section on further machines; the report |
| A4 | Could somebody else rebuild this from what you wrote? | The versions are pinned, the configuration files are in the report or beside it, the model identifiers are exact, and the commands are the ones you actually ran. | The report's reproduction section |

## B. Capstone 1 — the hardware and model plan

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| B1 | Is every memory budget arithmetic rather than a guess? | Each model's weight memory is parameter count times bytes per parameter, and each key-value cache figure comes from the per-token formula in Part 4 with the layer and key-value head counts named. |
| B2 | Does the plan state the context length each budget assumes? | A budget without a context length is not a budget. Each row names the context you will actually configure, not the model's trained maximum. |
| B3 | Is a headroom allowance written down and justified? | Operating system, engine working memory, cache growth and any second model, each with a figure and a sentence. |
| B4 | Does every model in the plan have a role and a licence? | Role in one phrase, licence from the model card, and "gated" noted where an acceptance step is needed. |
| B5 | Are the risks specific? | "The 30B model does not fit at 32k context on the 64 GB machine, so the fallback is 16k" is a risk. "Memory might be tight" is not. |

## C. Capstone 2 — the inference service

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| C1 | Does every item on the checklist have the check that proves it? | Each row names the command that was run and what it printed, not an intention. |
| C2 | Is nothing reachable beyond the machine without authentication and TLS? | A listening-socket check and a request from another device, both recorded. |
| C3 | Do the dashboards answer a question you wrote down first? | The questions were listed before the panels were built, and each panel maps to one. |
| C4 | Has at least one alert been proved to fire? | A deliberately induced condition, the alert's state change, and the time it took. |
| C5 | Has a backup been restored, not just taken? | A restore into a scratch location, with the comparison that showed it was intact. |
| C6 | Is there a rollback you have actually performed? | An upgrade and a return to the previous version, with the evidence that the service worked afterwards. |

## D. Capstone 3 — the cluster or tiered deployment

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| D1 | Is there a baseline measured before the distributed configuration? | A single-machine number taken first, under the same load, with the same model file. |
| D2 | Is the comparison honest about what changed? | One variable at a time, or an explicit list of everything that differed. |
| D3 | Is network utilisation measured rather than assumed? | Bytes across the link, or the link's own counters, against the measured throughput from Part 18. |
| D4 | Is failure behaviour observed rather than described? | A machine taken away mid-request, and what the client saw. |
| D5 | Does the conclusion follow from the numbers, including when it is "do not do this"? | A configuration that lost is reported as having lost, with the measurement. |

## E. Capstone 4 — the improved model

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| E1 | Was the evaluation set built before the training run? | A dated file, with its item count, written before the first run record. |
| E2 | Is the noise floor measured? | The same configuration evaluated more than once, and the spread of that repeat stated, so that a difference smaller than it is reported as no difference. |
| E3 | Is the before-and-after comparison run on the same file with the same settings? | Same task file, same shots, same chat template, same sampling settings, same seeds, both recorded. |
| E4 | Are regressions reported? | The categories that got worse, with the numbers, and a sentence on whether the trade is acceptable for your use. |
| E5 | Is the exported model the one that was measured? | The quantised file's evaluation is reported separately from the unquantised adapter's, because they are different models. |
| E6 | Is decontamination recorded? | What the evaluation set was checked against, what overlap was found, and what was done about it. |

## F. Capstone 5 — the agentic workstation

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| F1 | Does the sandbox test pass by failing? | Each boundary attempt was refused, and there is a control check that the agent can still reach what it legitimately needs. |
| F2 | Is the setup script the thing you actually ran? | A fresh run of the script produced the working configuration, and the report names what it did not cover. |
| F3 | Does each agent's model choice have a reason with evidence? | A measurement or an observed failure, not a preference. |
| F4 | Are the agent evaluation numbers from more than one run? | Three repeats at minimum for any comparison between configurations, with the spread reported. |
| F5 | Is every MCP server's permission stated? | What each server can read, what it can change, and what would happen if its output were hostile. |

## G. Capstone 6 — the report

| # | Question | What a "yes" looks like |
| --- | --- | --- |
| G1 | Does the architecture diagram match what is running? | Every box exists; nothing running is missing from the diagram. |
| G2 | Does the cost model name its inputs? | Measured idle and load power, the price per unit of electricity, the aggregate output rate, and the utilisation assumption, each as a figure you can point at. |
| G3 | Is there a section on what did not work? | At least three things, each with what you tried and what the evidence was. |
| G4 | Is there a sentence about what changed in your understanding? | Something you believed at Part 5 and no longer believe, with the measurement that changed it. |
| G5 | Is the maintenance plan actionable by somebody else? | Dated tasks, the commands, and where the state lives. |
| G6 | Does the report say what it did not measure? | A list, not an apology, and it is the section you would read first if it were somebody else's report. |

---

## The count

| Section | "no" answers | The sentences they become |
| --- | --- | --- |
| A. Applies to everything | <n> | <one line each> |
| B. Hardware and model plan | <n> | <one line each> |
| C. Inference service | <n> | <one line each> |
| D. Cluster or tiered deployment | <n> | <one line each> |
| E. Improved model | <n> | <one line each> |
| F. Agentic workstation | <n> | <one line each> |
| G. Report | <n> | <one line each> |

<!--
A capstone with a dozen honest "no" answers written into the report is a better piece of
work than one with none and no evidence. The failure the rubric is built to catch is not
an incomplete platform; it is a complete-looking document whose numbers nobody, including
its author, can trace back to a measurement.
-->