# Deployment measurement — <the architecture you chose>

<!--
Purpose: the deliverable of Capstone 3. One architecture, chosen with a reason, measured
         against the monolithic baseline it has to beat, with the network utilisation, the
         failure behaviour and the honest answer about what a further machine would change.
Platform: all
Minimum memory: 16 GB for the single-machine tiered design; more as your chosen
         architecture requires, from the Capstone 1 plan
Assumes: the baseline from Part 9's load-test lab or Part 6's benchmark lab, the link
         measurements from Part 18's lab, and whichever of Parts 19, 20, 21 and 22 your
         architecture comes from. The design document from Part 22's project, if you wrote
         one, is the input to section 1.

The rule: the baseline is measured first, before the distributed configuration exists. A
baseline taken afterwards, on a machine whose caches are warm and whose settings have
drifted, is not a baseline. Delete every comment block, including this one.
-->

**Author:** <you> · **Date:** <YYYY-MM-DD> · **Architecture:**
<layer split across machines / prefill and decode split / single-machine tiered design>

---

## 1. The architecture, and why this one

**In one paragraph:** <what runs where, and what a request does from arrival to answer.>

**Why this one and not the other two:**

| Option | Why not, for me | The measurement or fact that decided it |
| --- | --- | --- |
| Layer split across machines | <e.g. the model I want fits on one machine> | <the fit arithmetic from Capstone 1> |
| Prefill and decode split | <e.g. my fastest link cannot move one request's cache in less time than the prefill it replaces> | <the link measurement and the arithmetic> |
| Single-machine tiered design | <e.g. chosen: one machine, engines by tier behind one gateway> | <the baseline measurement> |

<One of the three rows says "chosen". The other two are the argument, and they are what
distinguishes a decision from a default.>

**What you predicted before measuring:** <write it here, now, in one sentence with a
direction in it: faster, slower, the same, or "it will only be about capacity". Part 19's
challenge exists because the answer is frequently "slower" and that is a result.>

## 2. The baseline

<One machine, no split, no tier. The same model file, the same quantisation, the same
context length, the same load generator, the same prompt set. Record it before you build
anything else.>

| Field | Value |
| --- | --- |
| Machine | <name and track> |
| Engine and version | <exact> |
| Model and quantisation | <exact file> |
| Context configured | <n tokens> |
| Key-value cache at startup | <tokens the engine reported> |
| Load generator and settings | <requests, concurrency levels, prompt set, maximum tokens, temperature> |
| Notebook label | <the label the record carries> |
| Date | <YYYY-MM-DD> |

| Concurrency | Requests completed | Failed | Output tokens per second | Requests per second | Time to first token, median | Time per output token, median |
| --- | --- | --- | --- | --- | --- | --- |
| <1> | | | | | | |
| <n> | | | | | | |

## 3. The distributed or tiered measurement

<The same table, the same load, the same prompt set. If anything else changed, list it
under the table; a comparison in which two things changed measures neither.>

| Field | Value |
| --- | --- |
| Machines and their parts | <which machine holds what> |
| Engine, version, and the distribution mechanism | <exact> |
| Split or placement | <the layer split, the pool sizes, or the tier boundaries> |
| Everything that differs from the baseline besides the architecture | <list, or "nothing"> |
| Notebook label | <the label> |
| Date | <YYYY-MM-DD> |

| Concurrency | Requests completed | Failed | Output tokens per second | Requests per second | Time to first token, median | Time per output token, median |
| --- | --- | --- | --- | --- | --- | --- |
| <1> | | | | | | |
| <n> | | | | | | |

**What changed, and in which direction:** <one sentence per column that moved, naming the
two labels being compared. If the distributed configuration lost, say so here plainly;
that is the commonest honest outcome at home and Part 19's challenge explains why.>

**What the second machine bought:** <capacity, speed, both or neither. A model that did not
fit before and fits now is a capacity result, and it is the result most cluster work
actually produces.>

## 4. Network utilisation

| Link | Carries | Measured throughput available | Bytes moved during the run | Utilisation | How measured |
| --- | --- | --- | --- | --- | --- |
| <machine to machine> | <per-token / per-request / load-time> | <from Part 18's lab> | <interface counters before and after, or the engine's own figure> | <bytes ÷ time ÷ available> | <counter names or tool> |

**Arithmetic for one request's key-value cache:** <bytes per token from Capstone 1> ×
<typical prompt tokens> = <bytes>. **Time to move it at the measured throughput:**
<result>. **Prefill time for the same prompt, measured:** <label>.

<Those three figures together are the argument for or against moving a cache between
machines. State which way they point in one sentence.>

**If the link was never near its capacity:** <say so. A cluster that is slow while its
link is idle is a latency problem, not a bandwidth problem, and Part 19 names the
difference as the thing home clusters most often get wrong.>

## 5. Failure behaviour

<Take something away while the service is answering, and record what happened. Do this on
purpose, once per failure, with the service under a light load.>

| What you removed | How | What the client saw | What the logs said | Recovery | Time to recover |
| --- | --- | --- | --- | --- | --- |
| A worker machine | <powered off, cable pulled, process stopped> | <error, hang, partial answer> | | <automatic or manual> | |
| The link | <cable pulled, interface down> | | | | |
| One engine process | <stopped> | | | | |
| The gateway | <stopped> | | | | |

**The failure that surprised you:** <one paragraph. There is usually one, and it is
usually a hang rather than an error.>

**What you changed as a result:** <a timeout, a health check, a retry, a supervision
rule, or nothing with a reason.>

## 6. What a further machine would change

| Field | Value |
| --- | --- |
| The machine | <a specific machine you could actually buy, with its memory and its track> |
| The role it would take | <from the six> |
| The link it would need | <class, and why that class> |
| What it would let you run that you cannot run now | <a model, a context length, a concurrency> |
| The measurement that motivates it | <a label from your notebook> |
| What it would not fix | <the thing people expect a machine to fix that it does not> |

<Fill this in even if your design is one machine. Especially then: it is the section that
turns "I only have one computer" into a design with a stated growth path, and it is the
answer to the question the rubric asks about the single-machine variant.>

## 7. What this measurement does not establish

<What you did not measure, what is arithmetic rather than measurement, what you took from
documentation without verifying, and where your two configurations were not strictly
comparable. Three items is a minimum.>

<Include the run-to-run variation you observed, or say that you ran each configuration
once and therefore cannot separate a difference from a wobble. Both are honest; only one
of them is a measurement.>