Capstone 3: Cluster or Tiered Deployment

Validated on: written from the documentation cited above and from Parts 18 to 22; not yet validated on hardware on any track. The measurements this project asks for are yours, and the comparison it insists on is the one that decides whether the architecture was worth building.

Objective

By the end of this project you will have one deployment architecture running, measured against the single-machine baseline it has to beat, with the link watched while it ran, the failure behaviour observed on purpose rather than imagined, and a written answer to what a further machine would change.

The result this project most often produces is that the distributed arrangement is slower than the single machine and worth having anyway, because it runs something that would not otherwise run at all. Part 19’s challenge exists for that reason. A capstone that reports that outcome, with the two measurements beside each other, is a better piece of work than one that reports a speed-up it cannot substantiate.

Architecture

Three options. Choose one, and the other two become the argument for why.

Option A: a layer split across machines

The model does not fit on any single machine you own, so its layers are distributed and each machine holds some of them. Every token passes through every machine in turn, so the link carries per-token traffic and latency rather than bandwidth is what you feel. This is Part 19’s lab at capstone scale, and on the NVIDIA and Apple tracks it also has native forms in Parts 20 and 21.

Option A — one model, layers spread across machines

• clientClientsthrough the gateway from Capstone 2
• routerHost machineruns the server process and holds the first share of the layers
• workerWorker Aholds a share of the layers, proportional to its memory
• workerWorker Boptional; the same again
• storageModel libraryeach machine reads the weights it needs at load time

• Clients ⟷ connected to Host machineclient traffic; the house network
• Host machine ⟷ connected to Worker Aper-token traffic; every token crosses it
• Worker A ⟷ connected to Worker Bper-token traffic
• Model library ⟷ connected to Host machineload time only
• Model library ⟷ connected to Worker Aload time only

What makes this option pay is capacity, not speed: a model that did not fit now runs. The per-token links are traversed for every token of every answer, which is why Part 19 concluded that a model that fits on one machine runs faster there.

⛨Security

The llama.cpp RPC backend’s own README is explicit: it is a proof-of-concept, it is fragile and insecure, and it should never be run on an open network or in a sensitive environment. That is a direct constraint on this option. Run the worker processes on a private link between machines you control, never exposed to the house network, and record in your document that you did.

Option B: prefill and decode on different machines

Reading the prompt and writing the answer want different hardware, so they run in separate pools and the key-value state moves between them. The link carries per-request traffic: one request’s cache, once. This is Part 22’s territory, and the arithmetic in Part 22 decides whether it can help you before you build it.

Option B — prefill machines and decode machines

• clientClientsthrough the gateway from Capstone 2
• routerAuthenticated gatewaykeys, limits, names; the only thing clients reach
• routerPhase-aware proxythe engine's own proxy, which knows the two pools apart
• prefillPrefill poolthe compute-rich machine; holds a prompt only while it reads it
• decodeDecode poolthe memory-rich machine; holds every conversation until it finishes
• cacheCache tierdevice memory, then host memory, then a shared store

• Clients ⟷ connected to Authenticated gatewayclient traffic, authenticated
• Authenticated gateway ⟷ connected to Phase-aware proxythe only hop that knows about phases
• Phase-aware proxy ⟷ connected to Prefill poolthe prompt
• Phase-aware proxy ⟷ connected to Decode poolthe continuation
• Prefill pool ⟷ connected to Cache tierwrites the request's key-value blocks
• Cache tier ⟷ connected to Decode poolreads them; per-request traffic

This option is honest only when one request's key-value cache moves across the link in less time than the prefill it replaced. That is arithmetic you can do before building anything, and at home the answer is usually no unless the link is very fast.

Option C: a single-machine tiered design

Several engines at different sizes behind one gateway, with a cache tier underneath, on one machine. A small model answers the cheap requests, a large one answers the expensive ones, and the routing rule decides. This is the single-machine variant, and on a house network it is frequently the correct answer rather than the fallback.

Option C — one machine, several tiers behind one gateway

• clientClientschat, an editor, an agent loop
• routerGateway with a routing rulechooses the tier; keys, limits and names as in Capstone 2
• workerSmall modelclassification, routing, short answers; resident at all times
• workerLarge modelthe expensive requests; loaded on demand by the model switch
• workerEmbedding modelretrieval; small and always resident
• cachePrompt cache tierreused prefixes kept in memory, spilled to disk

• Clients ⟷ connected to Gateway with a routing ruleclient traffic, authenticated
• Gateway with a routing rule ⟷ connected to Small modelloopback; no network at all
• Gateway with a routing rule ⟷ connected to Large modelloopback
• Gateway with a routing rule ⟷ connected to Embedding modelloopback
• Large model ⟷ connected to Prompt cache tierreads and writes reused prefixes

Nothing crosses a network here, which removes the entire class of problem the other two options are mostly about. What replaces it is a memory schedule: which models are resident together, which are loaded on demand, and what the switch costs when it happens mid-conversation.

⌂Single-machine path

Option C is a complete submission. It is not the consolation prize. What it owes, like every other option, is a baseline it is measured against and the paragraph about a further machine: which role would move first, which link class it would need, and which of your own measurements says so. A capstone that concludes “one machine, tiered, and here is the arithmetic that says a second one would not help until my concurrency triples” has done the harder piece of thinking.

Requirements

Every track needs the service from Capstone 2, the link measurements from Part 18’s lab, the load generator from Part 9’s lab, and about two hours. The memory floor is 16 GB for Option C on one machine; Options A and B need whatever your chosen model and split require, which your Capstone 1 fit table already says.

Track S — NVIDIA DGX Spark

All three options are open. With a pair on the direct high-speed link this is the one track where Option B’s arithmetic can plausibly come out in favour of splitting the phases, so if you have a pair, your document should say explicitly whether it does on your measurements and at what prompt length the answer changes. Option A over the same link is the vendor-documented path and Part 20 covers it.

Track X — AMD Ryzen AI Max+ 395Partial

These machines are usually memory-rich and network-poor, so the per-token and per-request options are constrained by the link rather than by the machine.

Option A works across two of these machines and Part 19’s lab is the reference, but with a 2.5 gigabit-class link the per-token traffic is the constraint and you should expect to report a capacity gain rather than a speed gain. Option B is hard to justify here on the arithmetic. Option C is the natural fit: a large unified memory holding several tiers at once is exactly what this machine is good at.

Track M — Apple siliconPartial

The two-machine paths need a second Mac and a Thunderbolt-class link; the course's own validation hardware has one Mac, so those pages are written from documentation.

Option A has a native form in Part 21 over Thunderbolt if you have two Macs. With one Mac, Option C is the path, and the single-machine forms in Part 21’s lab let you rehearse the mechanism with two ranks on one machine. Record which of those you did, because a mechanism check is not a cluster measurement and the document should not read as though it were.

Track N — NVIDIA desktop or laptop

All three options are open, and this is the track where Option C most often wins on arithmetic, because device memory is scarce and the model switch is what makes several tiers possible at all. If you have two cards in one machine, vLLM’s parallelism documentation is the reference for sizing tensor parallelism inside a node, and it is worth reading its note that without a fast interconnect between the cards, pipeline parallelism may serve you better than tensor parallelism.

Tasks

Working directory and terminal roles

Prepare the course execution workspace once before this procedure. It includes this part's scripts, data and shared Python helpers. In the client or training terminal, select this directory:

RunnableAll tracks

select this part’s execution directory

export LABS_ROOT="${LABS_ROOT:-$HOME/llm-course/labs}"
export LAB_DIR="$LABS_ROOT/capstone"
cd "$LAB_DIR"
pwd
test -f "deployment-measurement-template.md"

Expected result: pwd ends in capstone and the file check returns successfully. If it does not, finish workspace preparation before continuing. Activate the environment in the requirements for your track. Bare script and data filenames below are relative to this directory; paths to earlier experiments must point at the artefacts you actually retained.

Keep each foreground server in a separate terminal and send requests from this terminal. Reapply lesson-specific environment variables in each new shell. Stop at the first failed checkpoint and retain its output; the execution guide explains how to distinguish missing files, endpoint failures and capacity problems.

1. Choose the architecture, and write down what you expect

Open the template and fill in section 1: the option you chose, and the two rows saying why not the other two, each pointing at a measurement or at the fit arithmetic from Capstone 1.

Fragment — not complete on its own

deployment-measurement-template.md

# Deployment measurement — <the architecture you chose>

<!--
Purpose: the deliverable of Capstone 3. One architecture, chosen with a reason, measured
         against the monolithic baseline it has to beat, with the network utilisation, the
         failure behaviour and the honest answer about what a further machine would change.
Platform: all
Minimum memory: 16 GB for the single-machine tiered design; more as your chosen
         architecture requires, from the Capstone 1 plan
Assumes: the baseline from Part 9's load-test lab or Part 6's benchmark lab, the link
         measurements from Part 18's lab, and whichever of Parts 19, 20, 21 and 22 your
         architecture comes from. The design document from Part 22's project, if you wrote
         one, is the input to section 1.

The rule: the baseline is measured first, before the distributed configuration exists. A
baseline taken afterwards, on a machine whose caches are warm and whose settings have
drifted, is not a baseline. Delete every comment block, including this one.
-->

**Author:** <you> · **Date:** <YYYY-MM-DD> · **Architecture:**
<layer split across machines / prefill and decode split / single-machine tiered design>

---

## 1. The architecture, and why this one

**In one paragraph:** <what runs where, and what a request does from arrival to answer.>

**Why this one and not the other two:**

| Option | Why not, for me | The measurement or fact that decided it |
| --- | --- | --- |
| Layer split across machines | <e.g. the model I want fits on one machine> | <the fit arithmetic from Capstone 1> |
| Prefill and decode split | <e.g. my fastest link cannot move one request's cache in less time than the prefill it replaces> | <the link measurement and the arithmetic> |
| Single-machine tiered design | <e.g. chosen: one machine, engines by tier behind one gateway> | <the baseline measurement> |

<One of the three rows says "chosen". The other two are the argument, and they are what
distinguishes a decision from a default.>

**What you predicted before measuring:** <write it here, now, in one sentence with a
direction in it: faster, slower, the same, or "it will only be about capacity". Part 19's
challenge exists because the answer is frequently "slower" and that is a result.>

## 2. The baseline

<One machine, no split, no tier. The same model file, the same quantisation, the same
context length, the same load generator, the same prompt set. Record it before you build
anything else.>

| Field | Value |
| --- | --- |
| Machine | <name and track> |
| Engine and version | <exact> |
| Model and quantisation | <exact file> |
| Context configured | <n tokens> |
| Key-value cache at startup | <tokens the engine reported> |
| Load generator and settings | <requests, concurrency levels, prompt set, maximum tokens, temperature> |
| Notebook label | <the label the record carries> |
| Date | <YYYY-MM-DD> |

| Concurrency | Requests completed | Failed | Output tokens per second | Requests per second | Time to first token, median | Time per output token, median |
| --- | --- | --- | --- | --- | --- | --- |
| <1> | | | | | | |
| <n> | | | | | | |

## 3. The distributed or tiered measurement

<The same table, the same load, the same prompt set. If anything else changed, list it
under the table; a comparison in which two things changed measures neither.>

| Field | Value |
| --- | --- |
| Machines and their parts | <which machine holds what> |
| Engine, version, and the distribution mechanism | <exact> |
| Split or placement | <the layer split, the pool sizes, or the tier boundaries> |
| Everything that differs from the baseline besides the architecture | <list, or "nothing"> |
| Notebook label | <the label> |
| Date | <YYYY-MM-DD> |

| Concurrency | Requests completed | Failed | Output tokens per second | Requests per second | Time to first token, median | Time per output token, median |
| --- | --- | --- | --- | --- | --- | --- |
| <1> | | | | | | |
| <n> | | | | | | |

**What changed, and in which direction:** <one sentence per column that moved, naming the
two labels being compared. If the distributed configuration lost, say so here plainly;
that is the commonest honest outcome at home and Part 19's challenge explains why.>

**What the second machine bought:** <capacity, speed, both or neither. A model that did not
fit before and fits now is a capacity result, and it is the result most cluster work
actually produces.>

## 4. Network utilisation

| Link | Carries | Measured throughput available | Bytes moved during the run | Utilisation | How measured |
| --- | --- | --- | --- | --- | --- |
| <machine to machine> | <per-token / per-request / load-time> | <from Part 18's lab> | <interface counters before and after, or the engine's own figure> | <bytes ÷ time ÷ available> | <counter names or tool> |

**Arithmetic for one request's key-value cache:** <bytes per token from Capstone 1> ×
<typical prompt tokens> = <bytes>. **Time to move it at the measured throughput:**
<result>. **Prefill time for the same prompt, measured:** <label>.

<Those three figures together are the argument for or against moving a cache between
machines. State which way they point in one sentence.>

**If the link was never near its capacity:** <say so. A cluster that is slow while its
link is idle is a latency problem, not a bandwidth problem, and Part 19 names the
difference as the thing home clusters most often get wrong.>

## 5. Failure behaviour

<Take something away while the service is answering, and record what happened. Do this on
purpose, once per failure, with the service under a light load.>

| What you removed | How | What the client saw | What the logs said | Recovery | Time to recover |
| --- | --- | --- | --- | --- | --- |
| A worker machine | <powered off, cable pulled, process stopped> | <error, hang, partial answer> | | <automatic or manual> | |
| The link | <cable pulled, interface down> | | | | |
| One engine process | <stopped> | | | | |
| The gateway | <stopped> | | | | |

**The failure that surprised you:** <one paragraph. There is usually one, and it is
usually a hang rather than an error.>

**What you changed as a result:** <a timeout, a health check, a retry, a supervision
rule, or nothing with a reason.>

## 6. What a further machine would change

| Field | Value |
| --- | --- |
| The machine | <a specific machine you could actually buy, with its memory and its track> |
| The role it would take | <from the six> |
| The link it would need | <class, and why that class> |
| What it would let you run that you cannot run now | <a model, a context length, a concurrency> |
| The measurement that motivates it | <a label from your notebook> |
| What it would not fix | <the thing people expect a machine to fix that it does not> |

<Fill this in even if your design is one machine. Especially then: it is the section that
turns "I only have one computer" into a design with a stated growth path, and it is the
answer to the question the rubric asks about the single-machine variant.>

## 7. What this measurement does not establish

<What you did not measure, what is arithmetic rather than measurement, what you took from
documentation without verifying, and where your two configurations were not strictly
comparable. Three items is a minimum.>

<Include the run-to-run variation you observed, or say that you ran each configuration
once and therefore cannot separate a difference from a wobble. Both are honest; only one
of them is a measurement.>

Download deployment-measurement-template.md151 lines

RunnableAll tracks

start from the template

cp deployment-measurement-template.md deployment.md

Then write the prediction, in one sentence with a direction in it: faster, slower, the same, or “only about capacity”. Write it before you measure anything, and do not edit it afterwards.

This is the same discipline Part 19’s challenge and Part 22’s project both apply, and it exists because the temptation at the end of this project is to describe what happened as what you expected. A prediction on the page removes the temptation by making the difference visible and, frequently, interesting.

2. Measure the baseline, before the new architecture exists

One machine, no split, no tier. The same model file, the same quantisation, the same context length, the same load generator, the same prompt set, at the same concurrency levels you will use later.

Fragment — not complete on its own

python3 load-test.py \
  --base-url "$GATEWAY/v1" \
  --model "<the gateway name for the model under test>" \
  --concurrency 1,5,10,20 \
  --requests 40 \
  --label baseline \
  --engine "<engine>" \
  --engine-version "<version>" \
  --labbook labbook.md

Record the startup figures too: the key-value cache size, the maximum concurrency the engine reports at your context, and the peak memory. Those are the capacity of the baseline, and the comparison in section 3 is about capacity at least as much as it is about speed.

!Important

The baseline is measured first, on purpose. A baseline taken afterwards, on a machine whose caches are warm and whose settings have drifted since you started reconfiguring, is not a baseline. This is the single most common way a home cluster comparison becomes meaningless, and it costs nothing to avoid.

3. Build the architecture you chose

Follow the part that owns your option: Part 19 for a layer split with llama.cpp, Part 20 or 21 for the platform-native forms, Part 22 for the phase split and the cache tiers, and Capstone 2’s gateway for the tiered single-machine design.

Two things to write into section 3 as you build, because they are hard to reconstruct afterwards. First, the split or placement you chose and why: the proportions across machines, the pool sizes, or the tier boundaries. Second, everything that differs from the baseline besides the architecture itself. If you also changed the quantisation, the context length or the engine version, the comparison measures the sum of those changes and the document has to say so.

✕Common mistake

Changing two things at once is the error that makes the rest of this project unusable. It happens innocently: you rebuild the engine for the cluster and get a newer version, or you shorten the context because the split machine has less memory. Either is fine as long as the list under section 3’s table names it. An unlisted second change turns a measurement into a story.

4. Measure the new arrangement under the same load

The same command, the same concurrency levels, the same prompt set, a different label.

Then fill in the sentence that the whole project turns on: what changed, in which direction, naming the two labels. If the distributed configuration lost, say so plainly. Part 19’s challenge lists six reasons it commonly does, and the first of them is the one worth checking before anything else: a model that fits on one machine runs faster there.

Then the second sentence: what the second machine bought. Capacity, speed, both or neither. A model that did not fit before and fits now is a capacity result, and it is the result that most home cluster work actually produces.

5. Watch the link while it runs

Section 4 of the template. For each link, the throughput available from Part 18’s measurement, the bytes that actually moved during the run, and the utilisation those two imply.

The arithmetic to write out is the one Part 22 built its project around: the bytes per token of key-value cache from your Capstone 1 plan, times a typical prompt length, gives one request’s cache; that over the measured link throughput gives the transfer time; and beside it goes the measured prefill time for the same prompt. Those three figures next to each other are the argument for or against moving a cache between machines, and they are worth writing down even for Option A, where they explain why the answer was to move layers instead.

If the link was never near its capacity while the system was slow, you have a latency problem rather than a bandwidth problem. Part 19 names that distinction as the one home clusters most often get wrong, and finding it in your own measurement is worth more than reading it here.

6. Break it on purpose

Section 5, and this is the part most likely to teach you something you did not expect.

Under a light load, remove one thing at a time and record what the client saw, what the logs said, whether it recovered by itself and how long it took. A worker machine powered off. The link interface taken down. One engine process stopped. The gateway stopped.

The commonest surprise is that the client hangs rather than receiving an error, sometimes for a very long time. If that is what you observe, decide what your service should do instead and configure it: a timeout, a health check, a retry limit. Then write what you changed, or write that you chose to leave it and why.

⚠Warning

Do this deliberately and while nobody else is using the service. Pulling a cable on a machine that is mid-run is safe for the software and irritating for the household, and an engine killed while writing a cache file to a shared store is one of the few ways to leave state that needs cleaning up. Tell people first, and do the network failures before the process failures.

7. Say what a further machine would change

Section 6, and it is required whatever your option and however many machines you have.

A specific machine you could actually buy, with its memory and its track. The role it would take, from Part 18’s six. The link class it would need, and why that class rather than a cheaper one. What it would let you run that you cannot run now: a model, a context length, a concurrency. The measurement in your own notebook that motivates it. And the thing it would not fix, which is usually the one people expect a machine to fix.

For Option C this is the section that carries the deliverable. For Options A and B it is the section that stops the document being a description of what you happen to own.

8. Write what the measurement does not establish

Section 7. What you did not measure, what is arithmetic rather than measurement, where the two configurations were not strictly comparable, and how many times you ran each.

Include the run-to-run variation you observed. If you ran each configuration once, say so and say that you therefore cannot separate a difference from a wobble. Part 16 puts the practical minimum at three runs with the mean and the range, and the same logic applies to a load test: three runs is not statistics, it is enough to tell “these two differ” from “this measurement moves about”.

Use a comparison sheet with explicit controls

Name the baseline and proposed architecture before launching the latter. Keep model identity, representation, prompt classes, answer limits and offered load constant where the capacity permits. If a larger model is the reason for clustering, include a quality comparison and label the model difference instead of presenting an equal-workload speed claim.

For each run, save actual placement, transport evidence, request outcomes and resource use. Use the same timing definitions across configurations. Test cancellation and the loss of one participant with disposable work, then verify the documented recovery returns the service to a usable state.

Explain the observed result with the memory and communication model from Parts 18–22. A cluster that enables capacity without improving speed can satisfy the requirement; a tiered design that adds latency without a compensating benefit should be rejected. Keep the rejected design’s evidence in the report. The single-machine variant is valid for its declared scope and should mark physical interconnect and independent-node tests not run. Finish with the measured decision, failure procedure and the workload change that would cause you to revisit the architecture.

Validation

• deployment.md has every section filled in, with no angle-bracket fields left.
• Section 1 names the chosen option and gives a reason against each of the other two.
• The prediction is written and unedited, and it is dated before the first measurement record in the notebook.
• The baseline record in labbook.md is timestamped before the first record from the new architecture.
• Both measurement tables use the same load generator settings, the same prompt set and the same concurrency levels, and everything else that differs is listed.
• The startup key-value cache size and peak memory are recorded for both configurations.
• Section 4 has a utilisation figure or the words “not measured” for every link, and the three-figure cache arithmetic is written out.
• Section 5 has at least three deliberate failures with what the client saw.
• Section 6 names a specific machine, a role, a link class and a measurement label.
• Section 7 lists at least three things the measurement does not establish, including how many runs each configuration got.

Expected outcome

Two tables of numbers with their context, and a paragraph that says which way the comparison went and why. The shape below is what the deliverable commits to; the values in your own copy come from your notebook.

Pending validationThe comparison this project produces — the shape, to be filled in with your own

Configuration	What it is	Capacity evidence	Speed evidence
Baseline	one machine, one engine, no split and no tier	key-value cache size at startup, divided by the context	load-test rows at each concurrency level
Chosen architecture	the option from section 1, with its split or tier boundaries	the model or context that now runs and did not before	load-test rows at the same concurrency levels
Difference	capacity, speed, both or neither	pending	pending
Link	what it carried and how hard it worked	bytes moved against measured throughput	transfer arithmetic beside measured prefill
Failure	what the client saw when a part was removed	pending	pending
One more machine	the machine, the role, the link and the measurement behind it	pending	pending

the machines in your Capstone 1 plan · the engine your architecture uses as recorded in your deployment document · the model under test, identical in both configurations, identical in both configurations · 8,192 tokens of context · 2026-09-09

Both rows must come from the same load generator settings and the same prompt set. Anything else that differed between them belongs in the list under section 3 of the document.

Three documents count as finished and they read very differently.

The capacity result. The distributed configuration is slower per token and runs a model that would not otherwise run. The document says so, with both tables, and the conclusion is that the second machine bought capability rather than speed.

The consolidated result. The measurements said the split was not worth it, so the service runs on one machine in tiers, and section 6 names the condition under which the answer would change. This is the commonest honest outcome on a house network.

The genuine speed-up. Rarer, and it needs a fast link and a workload whose shape suits the split. If this is your result, section 3’s list of everything else that changed is the part a sceptical reader will go to first, so make it complete.

Troubleshooting

The cluster is slower and you cannot see why. Work Part 19’s challenge list: the model fits on one machine; the traffic is going over the wrong interface; the maximum transmission unit does not match at both ends; a processor device crept into the split; the proportions are in the wrong order for the machines’ memory; the fast transport is configured and not actually in use. Each has a check that takes minutes.

The link is idle and the system is slow. Latency, not bandwidth. Every token waits for a round trip, and no amount of extra capacity on the link changes a round trip. This is the expected result for a per-token split over a general-purpose network, and reporting it is the correct outcome.

The two configurations will not run the same load. If the distributed one cannot hold the same concurrency, that is a finding and not an obstacle: report both at the concurrency each supports and say plainly that the comparison is not like for like above that level.

The failure test does nothing visible. Check that you removed something the request path actually uses. Stopping a worker that holds no layers, or a pool with no traffic routed to it, tests nothing. Send a request first, confirm which machine served it, then remove that one.

Numbers move between runs by more than the difference you are trying to measure. That is the finding. Run each configuration three times, report the mean and the range, and if the ranges overlap, the honest conclusion is that on your hardware the two are indistinguishable for this workload.

You cannot decide between two options. Write both into section 1 with the arithmetic for each, build the cheaper one, and put the experiment that would distinguish them into section 6 beside the extra machine. An undecided question that names its own test is a legitimate part of a design.

Cleanup

Stop the distributed configuration if it is not what you intend to keep running, and leave the service from Capstone 2 up, because Capstone 5 uses it. Remove any worker process listening on a private link, especially the llama.cpp RPC workers, whose own documentation asks you not to leave them exposed. Delete the load-test output files you do not need, keep the notebook records, and unmount anything you mounted for the experiment.

Write in the deployment document what is still running afterwards. A capstone platform with a forgotten worker process on it is how the memory arithmetic in Capstone 1 stops matching reality.

What you learned

• The baseline has to exist before the thing it measures. Taken afterwards it measures a drifted machine, and the whole comparison quietly stops meaning anything.
• Capacity and speed are different results and both are worth having. A model that now runs at all is the outcome most home cluster work actually produces.
• The link’s annotation predicts the outcome. Per-token traffic feels latency, per-request traffic is an arithmetic question you can answer before building, and load-time traffic barely matters.
• An idle link during a slow run is a diagnosis, not a mystery. It says the problem is round trips.
• Failure behaviour is a property of your design. Until you have removed something on purpose, you do not know whether your clients see an error or a very long wait.
• Deciding not to distribute is a result. Level 4 taught five ways to add a machine so that you could decide, with a measurement, whether to.

Record in the notebook: both measurement sets with their labels, the link utilisation, the three-figure cache arithmetic, what each deliberate failure produced, and one sentence comparing your prediction from task 1 with what you measured.

Check your understanding

Question 1. Why does this project insist the baseline is measured before the new architecture is built?

• To save time later
• Because a baseline taken afterwards measures a machine whose caches are warm and whose settings have drifted, which is the commonest way a home cluster comparison becomes meaningless
• Because the engine cannot be reconfigured back
• Because the load generator caches its prompts

Show the answer and why

Answer: Because a baseline taken afterwards measures a machine whose caches are warm and whose settings have drifted, which is the commonest way a home cluster comparison becomes meaningless

Reconfiguring for a cluster changes context lengths, engine builds, batch settings and sometimes the model file. Every one of those makes a later "baseline" a measurement of something else, and the drift is invisible in the numbers themselves.

Question 2. Your two-machine layer split is slower per token than one machine, but it runs a model that did not fit before. How should the document report this?

• As a failure of the architecture
• As a capacity result: the second machine bought capability rather than speed, with both measurement tables shown
• By omitting the speed comparison
• By retuning until the split is faster

Show the answer and why

Answer: As a capacity result: the second machine bought capability rather than speed, with both measurement tables shown

This is the outcome Part 19 predicts and the commonest honest result of home cluster work. Every token crosses the link, so latency accumulates; what the second machine bought is the ability to run the model at all. Reporting both numbers is what makes it a result rather than a disappointment.

Question 3. During the run, the link shows almost no utilisation and the system is slow. What does that indicate?

• The measurement tool is broken
• A latency problem rather than a bandwidth problem: every token waits for a round trip, which extra link capacity does not change
• The model is too large
• The cache tier is not working

Show the answer and why

Answer: A latency problem rather than a bandwidth problem: every token waits for a round trip, which extra link capacity does not change

Part 19 names this as the distinction home clusters most often get wrong. A per-token split sends small messages very frequently, so the round trip dominates and the link never fills. Buying a wider link does not help; shortening the path does.

Question 4. Which of these belong in the list of things that differed between the two configurations? Select all that apply.

• You rebuilt the engine for the cluster and it is a newer version
• You shortened the context length because the split machine has less memory
• You used a different quantisation of the same model
• You used the same load generator settings and the same prompt set

Show the answer and why

Answer: You rebuilt the engine for the cluster and it is a newer version, You shortened the context length because the split machine has less memory, You used a different quantisation of the same model

Each of the first three makes the comparison measure the sum of several changes rather than the architecture. None of them invalidates the work; leaving them out of the list does. The fourth is the thing that was deliberately held constant, which is why it does not belong on the list of differences.

Question 5. True or false: a single-machine tiered design cannot satisfy this deliverable, because the page requires more than one machine.

• True
• False

Show the answer and why

Answer: False

Option C is a complete submission and on a house network it is frequently the right answer. What it owes is the same as what the other options owe: a baseline measured first, the new arrangement measured under the same load, and a section naming the machine that would change the answer, the role it would take and the measurement behind it.

Check my answersClear