Lab: Prepare Your Machine

Validated on: written from the vendor documentation cited above, and not yet run end to end on hardware on any of the four tracks. Each track’s versions, and the date it was run, are recorded here by the validation pass.

Objective

Before executing, read the lab execution and evidence guide. Use this lesson's explicit working directories and track setup; keep each server in its own terminal. Record hardware validation as pass, fail or not run, with the evidence requested below.

By the end of this lab your machine will be ready for every remaining part of the course, and you will have written down what it is:

• the operating system, driver and compute stack in the table below updated or installed, with each version read back by a command;
• the GPU seen by a command you ran, and the amount of memory it may address measured, then on Tracks X and M raised to a limit you computed from the machine’s total;
• Git, the course’s Python environment and uv at the pinned version;
• Docker Engine with Compose, plus the NVIDIA runtime on Tracks S and N, proven by running a container;
• the hf CLI at Hugging Face CLI 1.30.0 · verified 2026-09-08 or later, with HF_HOME pointing at ~/models/hf inside the library Part 4 created, and any existing login carried across;
• a check script that reports summary: 0 MISSING, a before-and-after record of what changed, and one dated JSON line in labbook.md.

The last of those is the one people skip, and it is what every later measurement is compared against.

⌂Single-machine path

This is a single-machine lab on every track. If you own two Sparks or several Macs, prepare each of them separately with this lab; nothing here joins them, and Part 18 is where that happens.

The stack you are preparing

Every model you run in this course passes through the same six layers, and each fails in its own way. The tasks work down the table, and the check script in task 1 reports on each row.

Layer	Track S (DGX Spark)	Track X (Ryzen AI Max+ 395)	Track M (Apple silicon)	Track N (NVIDIA card)
Operating system and kernel	DGX OS 7.x, updated by NVIDIA	Ubuntu 24.04 with the OEM kernel AMD names	macOS 26	Ubuntu 24.04, or Windows 11 with WSL2 Ubuntu
Driver	NVIDIA driver, part of DGX OS	amdgpu, inside the kernel (no DKMS package)	part of macOS	NVIDIA driver from Ubuntu, or from NVIDIA’s Windows installer under WSL2
GPU memory ceiling	none below total: unified	TTM page limit, which sets the GTT size	iogpu.wired_limit_mb	the card’s VRAM
Compute runtime	CUDA 13.0 toolkit, part of DGX OS	Vulkan (Mesa RADV) for llama.cpp; ROCm 7.2.1 for PyTorch	Metal, part of macOS; MLX from Part 1	CUDA 13 toolkit, installed here
Containers	Docker and the NVIDIA runtime, preinstalled	Docker Engine with Compose	not used: engines run natively	Docker Engine, Compose and the NVIDIA runtime; Docker Desktop under WSL2
Your tools	uv, the Part 1 environment, hf, HF_HOME	same	same	same

What earlier parts left on the machine

This lab builds on three earlier labs, and changes only one thing any of them set up.

Part	What it left	What this lab does with it
Part 1	~/llm-course with a .venv holding PyTorch for your track, uv, and labbook.md with Machine and Environment sections	checks uv against the pin, appends a baseline, updates the Machine section
Part 2	huggingface_hub in the .venv; possibly a login saved to ~/.cache/huggingface/token; checkpoints under ~/llm-course/models/	copies the login into the new Hugging Face home; leaves ~/llm-course/models/ alone
Part 4	the model library ~/models/ with its README.md, verified GGUF files, hf 1.30.0 or later, and ~/models/hf reserved for the Hugging Face cache	creates ~/models/hf in that same library and points HF_HOME at it; touches nothing else in ~/models

Task 7 moves the Hugging Face home into this library, and with it your login.

Requirements

Every track needs an administrator account, an internet connection, and the Part 1 and Part 4 labs done. The pinned versions this lab checks against are uv 0.12.11 · verified 2026-09-08, Hugging Face CLI 1.30.0 · verified 2026-09-08 and Docker Engine current · verified 2026-09-08. The memory floor is 8 GB with no reduced path, because nothing here loads a model.

The time columns are planning figures for a reasonable connection, not measurements. Sizes marked “apt shows it” depend on how far behind your system is: apt prints Need to get ... of archives before it asks to continue, and that line is worth recording.

Track	Largest downloads	Disk to allow	Attended	Unattended	Restarts
S	DGX OS updates (apt shows it); the CUDA test image nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04, whose Docker Hub listing gives 3,904,944,943 bytes compressed for arm64	15 GB	30 min	20–45 min	1
X	OEM kernel, ROCm 7.2.1 packages, Mesa Vulkan, Docker Engine (apt shows each); the amd-debug-tools 0.2.21 wheel, 135,696 bytes	30 GB	35 min	30–60 min	2
M	the macOS update (Software Update shows it); Xcode command-line tools; Homebrew	15 GB	25 min	20–60 min	0 or 1
N	CUDA toolkit 13.4.1 from NVIDIA’s ubuntu2404 index: 65 packages, 3,443,385,698 bytes to download, 7.15 GB installed, before Ubuntu’s own dependencies; Docker Engine; NVIDIA Container Toolkit 1.20.0	20 GB	30 min	20–40 min	0 or 1

The Track N package figures are arithmetic over NVIDIA’s repository index for cuda-toolkit-13 and everything it depends on inside that repository, read on 2026-09-13; the WSL repository carried toolkit 13.3.1 on the same day.

Track S — NVIDIA DGX Spark

A DGX Spark Founders Edition on DGX OS. On 2026-09-13 NVIDIA’s release notes listed DGX OS 7.5.0, driver 580.159.03, CUDA toolkit 13.0.2 and kernel 6.17 as current, and the update guide notes that its instructions apply to the Founders Edition only: OEM systems on the GB10 board may update on a different schedule and through their vendor’s tools. Use the supplied power adapter, which the known-issues page ties to boot and shutdown problems.

Track X — AMD Ryzen AI Max+ 395

A Ryzen AI Max+ 395 machine running Ubuntu 24.04, the distribution AMD’s Ryzen installation page documents; its ROCm commands and kernel package names do not carry over to Ubuntu 26.04 or Fedora 43, so on those read AMD’s current compatibility matrix first and treat task 3 as unwritten for your system. On 2026-09-13 that matrix, for ROCm 10.0.0, listed the “AMD Ryzen AI Max+ 395 (Radeon 8060S) (gfx1151)” without a support-tier qualifier, on Ubuntu 26.04 and 24.04.4 with the inbox kernel driver; task 3 explains why this lab still installs 7.2.1. If the machine came with Windows, install Linux for this lab: the kernel, TTM and ROCm steps have no Windows equivalent, and every later Track X page that uses PyTorch assumes them.

Track M — Apple silicon

An Apple silicon Mac on macOS 26, the course’s target. macOS 15 is the floor for this lab, because MLX documents its wired-limit function as “only useful on macOS 15.0 or higher”.

Track N — NVIDIA desktop or laptop

A desktop or laptop with an NVIDIA card, running Ubuntu 24.04 natively, or Windows 11 with a WSL2 Ubuntu. On Windows, the two PowerShell blocks in task 2 run in Windows; every other command on this page runs inside the WSL2 Ubuntu terminal, exactly as on Linux.

Preflight

Run this on every track before task 1. It proves the Part 1 and Part 4 outputs exist, that you can use sudo, and that the Hugging Face Hub answers.

RunnableAll tracks

preflight

cd ~/llm-course
ls -d .venv labbook.md
ls ~/models/README.md
uname -m
df -h ~
sudo -v && echo "sudo ok"
curl -sSI https://huggingface.co | head -n 1

Output — what you should see

.venv  labbook.md
/home/you/models/README.md
aarch64            (x86_64 on Tracks X and N, arm64 on Track M)
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p2  9xxG  xxxG  xxxG  xx% /
sudo ok
HTTP/2 200

Four things must hold. Both ls lines print paths rather than No such file or directory; if the first fails, do Part 1’s lab, and if the second fails, do task 7 of Part 4’s lab. Avail is at least the disk figure for your track. On Track M, uname -m must print arm64: x86_64 means the terminal runs under Rosetta and must be replaced by a native one before anything else.

Tasks

1. Take a snapshot before you change anything

Each track has a check script. It changes nothing: it inspects every row of the stack table and prints one line per item, marked ok, MISSING (a later part will fail without it, and the line names the task that fixes it) or NOTE (worth reading, not blocking), then a summary line. It exits with status 1 while anything is MISSING. Download your track’s script with the link under the listing into ~/llm-course, then save its report.

Track S — NVIDIA DGX Spark

RunnableTrack S · DGX Spark

prepare-spark.sh

#!/usr/bin/env bash
# Purpose: check a DGX Spark against everything the rest of the course assumes: DGX OS,
#          the GPU driver, the CUDA toolkit, Docker with the NVIDIA runtime and Compose,
#          the toolchain, the Part 1 environment and notebook, the hf CLI and the model
#          storage; print one line per item and a summary. It changes nothing.
# Platform: spark (DGX Spark and the OEM systems on the same GB10 board, aarch64)
# Minimum memory: 8 GB
# Assumes: DGX OS; the course directory ~/llm-course from Part 1 (override with COURSE=)
#          and the model library ~/models from Part 4 (override with MODELS=)
#
# Usage: bash prepare-spark.sh [--help]
# Output: "ok" (in place), "MISSING" (a later part fails without it; the text names the
#         task that fixes it) or "NOTE" (read it, not blocking), then
#         "summary: N MISSING, M NOTE". Exit status 1 when anything is MISSING.
set -euo pipefail

MODELS="${MODELS:-$HOME/models}"
COURSE="${COURSE:-$HOME/llm-course}"
UV_PIN="0.12.11"
HF_MIN="1.30.0"
INSTALL_HINT="sudo apt install"

for arg in "$@"; do
  case "$arg" in
    -h|--help) sed -n '2,14p' "$0"; exit 0 ;;
    *) echo "prepare-spark: unknown argument '$arg' (the script only reports; try --help)" >&2; exit 2 ;;
  esac
done

MISSING=0
NOTES=0
have()    { command -v "$1" >/dev/null 2>&1; }
section() { printf '\n== %s\n' "$1"; }
ok()      { printf '   ok       %-20s %s\n' "$1" "$2"; }
missing() { MISSING=$((MISSING + 1)); printf '   MISSING  %-20s %s\n' "$1" "$2"; }
note()    { NOTES=$((NOTES + 1)); printf '   NOTE     %-20s %s\n' "$1" "$2"; }
# version_ge A B: true when version A >= version B
version_ge() {
  awk -v a="$1" -v b="$2" 'BEGIN {
    na = split(a, x, /[.-]/); nb = split(b, y, /[.-]/); n = (na > nb) ? na : nb
    for (i = 1; i <= n; i++) { xi = (i <= na) ? x[i] + 0 : 0; yi = (i <= nb) ? y[i] + 0 : 0
      if (xi > yi) exit 0; if (xi < yi) exit 1 }
    exit 0 }'
}
first_match() { awk -v pat="$1" '$0 ~ pat && !done { print; done = 1 }'; }

printf 'prepare-spark.sh on %s, %s\n' "$(hostname)" "$(date '+%Y-%m-%d %H:%M')"

section "1. Machine and operating system"
arch="$(uname -m)"
if [ "$arch" = "aarch64" ]; then
  ok "architecture" "$arch"
else
  missing "architecture" "$arch is not a GB10 machine: use the script for your own track"
fi
ok "kernel" "$(uname -r)"
if [ -r /etc/os-release ]; then
  # shellcheck disable=SC1091
  . /etc/os-release
  ok "distribution" "${PRETTY_NAME:-unknown}"
fi
mem_kib="$(awk '/^MemTotal:/ {print $2}' /proc/meminfo)"
ok "memory (MemTotal)" "$(awk -v k="$mem_kib" 'BEGIN {printf "%.1f GiB, shared by CPU and GPU", k / 1048576}')"

section "2. GPU and driver"
if have nvidia-smi; then
  gpu="$(nvidia-smi --query-gpu=name,driver_version --format=csv,noheader 2>/dev/null | first_match . || true)"
  if [ -n "$gpu" ]; then
    ok "GPU, driver" "$gpu"
    driver="${gpu##*, }"
    if ! version_ge "$driver" "580"; then
      note "driver" "$driver is older than the 580 series CUDA 13 needs: task 2 updates DGX OS"
    fi
  else
    missing "GPU" "nvidia-smi runs but lists no GPU: see Troubleshooting"
  fi
else
  missing "nvidia-smi" "not found; DGX OS ships the driver, so see the DGX Spark user guide"
fi

section "3. CUDA toolkit"
if have nvcc; then
  ok "nvcc" "$(nvcc --version | first_match release)"
elif [ -x /usr/local/cuda/bin/nvcc ]; then
  note "nvcc" "installed at /usr/local/cuda/bin but not on PATH: task 3"
else
  missing "nvcc" "not found; DGX OS includes the CUDA toolkit: task 2, then task 3"
fi

section "4. Containers"
if have docker; then
  ok "docker" "$(docker --version 2>/dev/null)"
  docker_cmd=""
  if docker info >/dev/null 2>&1; then
    docker_cmd="docker"
    ok "docker access" "'docker info' works for $(id -un)"
  elif sudo -n docker info >/dev/null 2>&1; then
    docker_cmd="sudo -n docker"
    note "docker access" "works with sudo only; fine on DGX OS, or join the docker group (task 6)"
  else
    note "docker access" "'docker info' failed for $(id -un) without a password; run as a docker group member (task 6)"
  fi
  if [ -n "$docker_cmd" ]; then
    # shellcheck disable=SC2016  # a Go template for docker, not a shell expansion
    runtimes="$($docker_cmd info --format '{{range $name, $rt := .Runtimes}}{{$name}} {{end}}' 2>/dev/null || true)"
    case " $runtimes " in
      *" nvidia "*) ok "nvidia runtime" "registered (runtimes: ${runtimes% })" ;;
      # Inferred, not observed on a Spark: NVIDIA's own check is the --gpus=all container
      *) note "nvidia runtime" "not listed (${runtimes% }); task 6's --gpus=all container test is the check, but Part 7's --runtime=nvidia needs it: Troubleshooting" ;;
    esac
    if compose="$($docker_cmd compose version --short 2>/dev/null)"; then
      ok "docker compose" "$compose"
    else
      missing "docker compose" "the Compose plugin is not installed; Part 7 needs it (task 6)"
    fi
  fi
else
  missing "docker" "not found, which is unusual on DGX OS: task 6"
fi
if have nvidia-ctk; then
  ok "nvidia-ctk" "$(nvidia-ctk --version 2>/dev/null | first_match .)"
else
  note "nvidia-ctk" "not found; the user guide says the container toolkit is preinstalled"
fi

section "5. Toolchain and the Part 1 course directory"
for tool in git python3 curl; do
  if have "$tool"; then
    ok "$tool" "$("$tool" --version 2>&1 | awk 'NR == 1 {print $1, $2, $3}')"
  else
    missing "$tool" "install it with: $INSTALL_HINT $tool"
  fi
done
if have uv; then
  uv_version="$(uv --version | awk '{print $2}')"
  if [ "$uv_version" = "$UV_PIN" ]; then ok "uv" "$uv_version (the course pin)"; else note "uv" "$uv_version; the course is pinned to $UV_PIN (task 5)"; fi
else
  missing "uv" "not found on PATH: task 5"
fi
if [ -x "$COURSE/.venv/bin/python" ]; then ok "course environment" "$COURSE/.venv"; else missing "course environment" "$COURSE/.venv not found: Part 1's lab, tasks 2 and 3"; fi
if [ -f "$COURSE/labbook.md" ]; then ok "lab notebook" "$COURSE/labbook.md"; else missing "lab notebook" "$COURSE/labbook.md not found: Part 1's lab, task 1"; fi

section "6. Hugging Face CLI, login and model storage"
hf_bin=""
if have hf; then hf_bin="$(command -v hf)"; elif [ -x "$COURSE/.venv/bin/hf" ]; then hf_bin="$COURSE/.venv/bin/hf"; fi
if [ -n "$hf_bin" ]; then
  hf_version="$("$hf_bin" version --format json 2>/dev/null | sed -n 's/.*"version": *"\([^"]*\)".*/\1/p')"
  if [ -n "$hf_version" ] && version_ge "$hf_version" "$HF_MIN"; then
    ok "hf" "$hf_version ($hf_bin)"
  else
    missing "hf" "version '${hf_version:-unknown}' is older than $HF_MIN: task 7"
  fi
else
  missing "hf" "not on PATH and not in $COURSE/.venv: task 7"
fi
if [ -d "$MODELS" ]; then ok "model library" "$MODELS"; else missing "model library" "$MODELS not found: Part 4's lab, task 7, or task 7 here"; fi
if [ -f "$MODELS/README.md" ]; then ok "library README" "$MODELS/README.md"; else note "library README" "missing: Part 4's lab, task 7, writes it"; fi
if [ -d "$MODELS/hf" ]; then ok "HF_HOME directory" "$MODELS/hf"; else missing "HF_HOME directory" "$MODELS/hf not found: task 7"; fi
if [ "${HF_HOME:-}" = "$MODELS/hf" ]; then
  ok "HF_HOME" "$HF_HOME"
else
  missing "HF_HOME" "is '${HF_HOME:-unset}' in this shell, expected $MODELS/hf: task 7"
fi
if [ -f "$MODELS/hf/token" ]; then
  ok "stored login" "$MODELS/hf/token"
elif [ -f "$HOME/.cache/huggingface/token" ]; then
  note "stored login" "only under ~/.cache/huggingface, which hf stops reading once HF_HOME moves: task 7"
else
  note "stored login" "none; needed only for gated models (task 7)"
fi
df_target="$MODELS"; [ -d "$df_target" ] || df_target="$HOME"
ok "free disk" "$(df -Pk "$df_target" | awk 'NR == 2 {printf "%d GiB", $4 / 1048576}') on the filesystem holding $df_target"

printf '\nsummary: %d MISSING, %d NOTE\n' "$MISSING" "$NOTES"
[ "$MISSING" -eq 0 ]

Download prepare-spark.sh175 lines

RunnableTrack S · DGX Spark

the starting state, saved to a file

cd ~/llm-course
bash prepare-spark.sh | tee part05-before.txt

Track X — AMD Ryzen AI Max+ 395

RunnableTrack X · Ryzen AI Max+

prepare-strix.sh

#!/usr/bin/env bash
# Purpose: check a Ryzen AI Max+ 395 (Strix Halo) machine against everything the rest of
#          the course assumes: the kernel AMD's Ryzen ROCm page names, the render and video
#          groups, how much memory the GPU may address (VRAM, GTT, the TTM page limit), the
#          Vulkan and ROCm paths, Docker with Compose, the toolchain, the Part 1 environment
#          and notebook, the hf CLI and the model storage. It changes nothing.
# Platform: strix (AMD Ryzen AI Max+ 395, gfx1151, on Linux; Ubuntu 24.04 is the path
#          AMD's Ryzen installation page documents)
# Minimum memory: 8 GB
# Assumes: Linux with the inbox amdgpu kernel driver; ~/llm-course from Part 1 (override
#          with COURSE=) and ~/models from Part 4 (override with MODELS=)
#
# Usage: bash prepare-strix.sh [--help]
# Output: "ok" (in place), "MISSING" (a later part fails without it; the text names the
#         task that fixes it) or "NOTE" (read it, not blocking), then
#         "summary: N MISSING, M NOTE". Exit status 1 when anything is MISSING.
set -euo pipefail

MODELS="${MODELS:-$HOME/models}"
COURSE="${COURSE:-$HOME/llm-course}"
UV_PIN="0.12.11"
HF_MIN="1.30.0"
INSTALL_HINT="sudo apt install"
PAGE_BYTES=4096   # the TTM limit is counted in pages of this size

for arg in "$@"; do
  case "$arg" in
    -h|--help) sed -n '2,16p' "$0"; exit 0 ;;
    *) echo "prepare-strix: unknown argument '$arg' (the script only reports; try --help)" >&2; exit 2 ;;
  esac
done

MISSING=0
NOTES=0
have()    { command -v "$1" >/dev/null 2>&1; }
section() { printf '\n== %s\n' "$1"; }
ok()      { printf '   ok       %-20s %s\n' "$1" "$2"; }
missing() { MISSING=$((MISSING + 1)); printf '   MISSING  %-20s %s\n' "$1" "$2"; }
note()    { NOTES=$((NOTES + 1)); printf '   NOTE     %-20s %s\n' "$1" "$2"; }
# version_ge A B: true when version A >= version B
version_ge() {
  awk -v a="$1" -v b="$2" 'BEGIN {
    na = split(a, x, /[.-]/); nb = split(b, y, /[.-]/); n = (na > nb) ? na : nb
    for (i = 1; i <= n; i++) { xi = (i <= na) ? x[i] + 0 : 0; yi = (i <= nb) ? y[i] + 0 : 0
      if (xi > yi) exit 0; if (xi < yi) exit 1 }
    exit 0 }'
}
first_match() { awk -v pat="$1" '$0 ~ pat && !done { print; done = 1 }'; }

printf 'prepare-strix.sh on %s, %s\n' "$(hostname)" "$(date '+%Y-%m-%d %H:%M')"

section "1. Machine, operating system and kernel"
ok "architecture" "$(uname -m)"
kernel="$(uname -r)"
VERSION_ID=""
if [ -r /etc/os-release ]; then
  # shellcheck disable=SC1091
  . /etc/os-release
  ok "distribution" "${PRETTY_NAME:-unknown}"
fi
# AMD's Ryzen ROCm page: "the 6.14-1018 OEM kernel or newer" on Ubuntu 24.04
kernel_num="$(printf '%s' "$kernel" | cut -d- -f1,2 | tr '-' '.')"
if [ "$VERSION_ID" = "24.04" ]; then
  if version_ge "$kernel_num" "6.14.0.1018" && case "$kernel" in *-oem) true ;; *) false ;; esac; then
    ok "kernel" "$kernel (OEM, 6.14-1018 or newer)"
  elif version_ge "$kernel_num" "6.14.0.1018"; then
    note "kernel" "$kernel is new enough but is not the OEM kernel AMD's Ryzen page names: task 2"
  else
    missing "kernel" "$kernel is older than the 6.14-1018 OEM kernel AMD requires for ROCm: task 2"
  fi
else
  note "kernel" "$kernel on ${PRETTY_NAME:-this distribution}; AMD's Ryzen page documents Ubuntu 24.04 only"
fi
mem_kib="$(awk '/^MemTotal:/ {print $2}' /proc/meminfo)"
ok "memory (MemTotal)" "$(awk -v k="$mem_kib" 'BEGIN {printf "%.2f GiB", k / 1048576}')"
for group in render video; do
  if id -nG | tr ' ' '\n' | grep -qx "$group"; then
    ok "group $group" "$(id -un) is a member"
  else
    missing "group $group" "$(id -un) is not a member (log out and in after adding): task 3"
  fi
done

section "2. How much memory the GPU may address"
if have lspci; then
  ok "PCI display device" "$(lspci | awk 'tolower($0) ~ /vga|display/ && !done { sub(/^[^ ]+ /, ""); print; done = 1 }')"
fi
found_card="no"
for dev in /sys/class/drm/card*/device; do
  [ -r "$dev/mem_info_gtt_total" ] || continue
  found_card="yes"
  card="$(basename "$(dirname "$dev")")"
  vram="$(cat "$dev/mem_info_vram_total")"
  gtt="$(cat "$dev/mem_info_gtt_total")"
  ok "$card VRAM" "$(awk -v b="$vram" 'BEGIN {printf "%.2f GiB (the firmware carve-out)", b / 1073741824}')"
  ok "$card GTT" "$(awk -v b="$gtt" 'BEGIN {printf "%.2f GiB (system memory the GPU may map)", b / 1073741824}')"
  if [ "$vram" -gt 644245094 ]; then   # 0.6 GiB
    note "$card VRAM" "AMD recommends the minimum carve-out (0.5 GB) in the firmware settings, and a larger TTM limit"
  fi
done
[ "$found_card" = "yes" ] || missing "amdgpu" "no card exposes mem_info_gtt_total: the amdgpu driver is not loaded"
if [ -r /sys/module/ttm/parameters/pages_limit ]; then
  pages="$(cat /sys/module/ttm/parameters/pages_limit)"
  limit_line="$(awk -v p="$pages" -v s="$PAGE_BYTES" -v k="$mem_kib" \
    'BEGIN {g = p * s / 1073741824; printf "%d pages = %.2f GiB = %.0f%% of MemTotal", p, g, 100 * g / (k / 1048576)}')"
  percent="$(awk -v p="$pages" -v s="$PAGE_BYTES" -v k="$mem_kib" 'BEGIN {printf "%d", 100 * p * s / 1024 / k}')"
  if [ "$percent" -le 55 ]; then
    note "TTM pages_limit" "$limit_line: still about the kernel default of half; task 4 raises it"
  else
    ok "TTM pages_limit" "$limit_line"
  fi
else
  note "TTM pages_limit" "/sys/module/ttm/parameters/pages_limit not readable"
fi
if [ -f /etc/modprobe.d/ttm.conf ]; then
  ok "ttm.conf" "$(cat /etc/modprobe.d/ttm.conf)"
fi

section "3. The two GPU software paths"
if have vulkaninfo; then
  summary="$(vulkaninfo --summary 2>/dev/null || true)"
  names="$(printf '%s\n' "$summary" | awk -F'= ' '/deviceName/ {printf "%s%s", sep, $2; sep = "; "}')"
  drivers="$(printf '%s\n' "$summary" | awk -F'= ' '/driverName/ {printf "%s%s", sep, $2; sep = "; "}')"
  if [ -n "$names" ]; then
    ok "Vulkan devices" "$names"
    ok "Vulkan drivers" "${drivers:-not reported}"
  else
    missing "Vulkan devices" "vulkaninfo --summary lists no device: task 3"
  fi
else
  missing "vulkaninfo" "not found: task 3 installs mesa-vulkan-drivers and vulkan-tools"
fi
if have rocminfo; then
  gfx="$(rocminfo 2>/dev/null | awk '$1 == "Name:" && $2 ~ /^gfx/ && !done {print $2; done = 1}')"
  if [ -n "$gfx" ]; then
    ok "ROCm agent" "$gfx"
  else
    missing "ROCm agent" "rocminfo lists no gfx agent: groups, then a reboot (task 3)"
  fi
  if have amd-smi; then ok "amd-smi" "$(amd-smi version 2>/dev/null | first_match .)"; fi
else
  note "ROCm" "rocminfo not found: ROCm is not installed (task 3; PyTorch on the GPU needs it)"
fi
if have dpkg-query && dpkg-query -W -f='${db:Status-Status}\n' amdgpu-dkms 2>/dev/null | grep -qx installed; then
  missing "amdgpu-dkms" "installed, but AMD's Ryzen page requires the inbox driver: Troubleshooting"
fi

section "4. Containers"
if have docker; then
  ok "docker" "$(docker --version 2>/dev/null)"
  if docker info >/dev/null 2>&1; then
    ok "docker access" "'docker info' works for $(id -un)"
  else
    missing "docker access" "'docker info' failed for $(id -un): not in the docker group yet (task 6)"
  fi
  if compose="$(docker compose version --short 2>/dev/null)"; then
    ok "docker compose" "$compose"
  else
    missing "docker compose" "the Compose plugin is not installed; Part 7 needs it (task 6)"
  fi
elif have podman; then
  note "podman" "$(podman --version 2>/dev/null); Part 7's lab is written for Docker Engine with Compose"
else
  missing "docker" "neither docker nor podman found: task 6"
fi
if [ -e /dev/kfd ]; then
  ok "/dev/kfd" "present (ROCm opens it; Part 7 passes it into containers)"
else
  note "/dev/kfd" "absent: ROCm and ROCm containers cannot reach the GPU without it (Troubleshooting)"
fi

section "5. Toolchain and the Part 1 course directory"
for tool in git python3 curl; do
  if have "$tool"; then
    ok "$tool" "$("$tool" --version 2>&1 | awk 'NR == 1 {print $1, $2, $3}')"
  else
    missing "$tool" "install it with: $INSTALL_HINT $tool"
  fi
done
if have uv; then
  uv_version="$(uv --version | awk '{print $2}')"
  if [ "$uv_version" = "$UV_PIN" ]; then ok "uv" "$uv_version (the course pin)"; else note "uv" "$uv_version; the course is pinned to $UV_PIN (task 5)"; fi
else
  missing "uv" "not found on PATH: task 5"
fi
if [ -x "$COURSE/.venv/bin/python" ]; then ok "course environment" "$COURSE/.venv"; else missing "course environment" "$COURSE/.venv not found: Part 1's lab, tasks 2 and 3"; fi
if [ -f "$COURSE/labbook.md" ]; then ok "lab notebook" "$COURSE/labbook.md"; else missing "lab notebook" "$COURSE/labbook.md not found: Part 1's lab, task 1"; fi

section "6. Hugging Face CLI, login and model storage"
hf_bin=""
if have hf; then hf_bin="$(command -v hf)"; elif [ -x "$COURSE/.venv/bin/hf" ]; then hf_bin="$COURSE/.venv/bin/hf"; fi
if [ -n "$hf_bin" ]; then
  hf_version="$("$hf_bin" version --format json 2>/dev/null | sed -n 's/.*"version": *"\([^"]*\)".*/\1/p')"
  if [ -n "$hf_version" ] && version_ge "$hf_version" "$HF_MIN"; then
    ok "hf" "$hf_version ($hf_bin)"
  else
    missing "hf" "version '${hf_version:-unknown}' is older than $HF_MIN: task 7"
  fi
else
  missing "hf" "not on PATH and not in $COURSE/.venv: task 7"
fi
if [ -d "$MODELS" ]; then ok "model library" "$MODELS"; else missing "model library" "$MODELS not found: Part 4's lab, task 7, or task 7 here"; fi
if [ -f "$MODELS/README.md" ]; then ok "library README" "$MODELS/README.md"; else note "library README" "missing: Part 4's lab, task 7, writes it"; fi
if [ -d "$MODELS/hf" ]; then ok "HF_HOME directory" "$MODELS/hf"; else missing "HF_HOME directory" "$MODELS/hf not found: task 7"; fi
if [ "${HF_HOME:-}" = "$MODELS/hf" ]; then
  ok "HF_HOME" "$HF_HOME"
else
  missing "HF_HOME" "is '${HF_HOME:-unset}' in this shell, expected $MODELS/hf: task 7"
fi
if [ -f "$MODELS/hf/token" ]; then
  ok "stored login" "$MODELS/hf/token"
elif [ -f "$HOME/.cache/huggingface/token" ]; then
  note "stored login" "only under ~/.cache/huggingface, which hf stops reading once HF_HOME moves: task 7"
else
  note "stored login" "none; needed only for gated models (task 7)"
fi
df_target="$MODELS"; [ -d "$df_target" ] || df_target="$HOME"
ok "free disk" "$(df -Pk "$df_target" | awk 'NR == 2 {printf "%d GiB", $4 / 1048576}') on the filesystem holding $df_target"

printf '\nsummary: %d MISSING, %d NOTE\n' "$MISSING" "$NOTES"
[ "$MISSING" -eq 0 ]

Download prepare-strix.sh221 lines

RunnableTrack X · Ryzen AI Max+

the starting state, saved to a file

cd ~/llm-course
bash prepare-strix.sh | tee part05-before.txt

Track M — Apple silicon

RunnableTrack M · Apple silicon

prepare-mac.sh

#!/usr/bin/env bash
# Purpose: check an Apple silicon Mac against everything the rest of the course assumes:
#          a native arm64 shell, the macOS version, the chip and its GPU, the wired-memory
#          limit and what MLX reports about it, the Xcode command-line tools and Homebrew,
#          the toolchain, the Part 1 environment and notebook, the hf CLI and the model
#          storage. It changes nothing.
# Platform: mac (Apple silicon; macOS 26 is the course target, 15 is the floor for the
#          wired-limit steps)
# Minimum memory: 8 GB
# Assumes: ~/llm-course from Part 1 with MLX in its .venv (override with COURSE=) and
#          ~/models from Part 4 (override with MODELS=)
#
# Usage: bash prepare-mac.sh [--help]
# Output: "ok" (in place), "MISSING" (a later part fails without it; the text names the
#         task that fixes it) or "NOTE" (read it, not blocking), then
#         "summary: N MISSING, M NOTE". Exit status 1 when anything is MISSING.
set -euo pipefail

MODELS="${MODELS:-$HOME/models}"
COURSE="${COURSE:-$HOME/llm-course}"
UV_PIN="0.12.11"
HF_MIN="1.30.0"
INSTALL_HINT="brew install"

for arg in "$@"; do
  case "$arg" in
    -h|--help) sed -n '2,16p' "$0"; exit 0 ;;
    *) echo "prepare-mac: unknown argument '$arg' (the script only reports; try --help)" >&2; exit 2 ;;
  esac
done

MISSING=0
NOTES=0
have()    { command -v "$1" >/dev/null 2>&1; }
section() { printf '\n== %s\n' "$1"; }
ok()      { printf '   ok       %-20s %s\n' "$1" "$2"; }
missing() { MISSING=$((MISSING + 1)); printf '   MISSING  %-20s %s\n' "$1" "$2"; }
note()    { NOTES=$((NOTES + 1)); printf '   NOTE     %-20s %s\n' "$1" "$2"; }
# version_ge A B: true when version A >= version B
version_ge() {
  awk -v a="$1" -v b="$2" 'BEGIN {
    na = split(a, x, /[.-]/); nb = split(b, y, /[.-]/); n = (na > nb) ? na : nb
    for (i = 1; i <= n; i++) { xi = (i <= na) ? x[i] + 0 : 0; yi = (i <= nb) ? y[i] + 0 : 0
      if (xi > yi) exit 0; if (xi < yi) exit 1 }
    exit 0 }'
}
first_match() { awk -v pat="$1" '$0 ~ pat && !done { print; done = 1 }'; }

printf 'prepare-mac.sh on %s, %s\n' "$(hostname)" "$(date '+%Y-%m-%d %H:%M')"

section "1. Machine and operating system"
arch="$(uname -m)"
if [ "$arch" = "arm64" ]; then
  ok "architecture" "$arch"
else
  missing "architecture" "$arch: this shell runs under Rosetta; open a native terminal"
fi
macos="$(sw_vers -productVersion)"
if version_ge "$macos" "26"; then
  ok "macOS" "$macos (build $(sw_vers -buildVersion))"
elif version_ge "$macos" "15"; then
  note "macOS" "$macos: works, but the course is written against macOS 26 (task 2)"
else
  missing "macOS" "$macos: MLX's wired-limit function needs 15.0 or later (task 2)"
fi
ok "chip" "$(sysctl -n machdep.cpu.brand_string 2>/dev/null || echo unknown)"
mem_bytes="$(sysctl -n hw.memsize)"
ok "memory (hw.memsize)" "$(awk -v b="$mem_bytes" 'BEGIN {printf "%d bytes = %.0f GiB", b, b / 1073741824}')"

section "2. GPU and the wired-memory limit"
display="$(system_profiler SPDisplaysDataType 2>/dev/null || true)"
ok "GPU" "$(printf '%s\n' "$display" | awk -F': ' '/Chipset Model/ && !done {print $2; done = 1}'), $(printf '%s\n' "$display" | awk -F': ' '/Total Number of Cores/ && !done {print $2 " cores"; done = 1}')"
wired="$(sysctl -n iogpu.wired_limit_mb 2>/dev/null || echo unavailable)"
if [ "$wired" = "unavailable" ]; then
  missing "iogpu.wired_limit_mb" "the sysctl does not exist on this macOS: task 2"
elif [ "$wired" = "0" ]; then
  note "iogpu.wired_limit_mb" "0, the system default: task 4 sets it"
else
  ok "iogpu.wired_limit_mb" "$(awk -v m="$wired" -v b="$mem_bytes" 'BEGIN {printf "%d MB = %.1f GiB = %.0f%% of memory", m, m / 1024, 100 * m * 1048576 / b}')"
fi
if [ -x "$COURSE/.venv/bin/python" ]; then
  mlx_line="$("$COURSE/.venv/bin/python" -c '
import mlx.core as mx
info = mx.device_info()
ws = info.get("max_recommended_working_set_size")
total = info.get("memory_size")
print(f"max_recommended_working_set_size {ws} bytes ({ws / 2**30:.1f} GiB) of memory_size {total} bytes" if ws and total else "")
' 2>/dev/null || true)"
  if [ -n "$mlx_line" ]; then ok "MLX device_info" "$mlx_line"; else note "MLX device_info" "mlx not importable from $COURSE/.venv: Part 1's lab installs it"; fi
fi

section "3. Command-line tools and Homebrew"
if clt="$(xcode-select -p 2>/dev/null)"; then
  ok "developer tools" "$clt"
else
  missing "developer tools" "not installed: task 3 (xcode-select --install)"
fi
if have brew; then
  ok "homebrew" "$(brew --version | first_match Homebrew) at $(brew --prefix)"
else
  missing "homebrew" "not found; Part 6 installs cmake with it: task 3"
fi

section "4. Containers"
ok "containers" "not used on this track: the course runs engines natively on macOS (task 6)"

section "5. Toolchain and the Part 1 course directory"
for tool in git python3 curl; do
  if have "$tool"; then
    ok "$tool" "$("$tool" --version 2>&1 | awk 'NR == 1 {print $1, $2, $3}')"
  else
    missing "$tool" "install it with: $INSTALL_HINT $tool"
  fi
done
if have uv; then
  uv_version="$(uv --version | awk '{print $2}')"
  if [ "$uv_version" = "$UV_PIN" ]; then ok "uv" "$uv_version (the course pin)"; else note "uv" "$uv_version; the course is pinned to $UV_PIN (task 5)"; fi
else
  missing "uv" "not found on PATH: task 5"
fi
if [ -x "$COURSE/.venv/bin/python" ]; then ok "course environment" "$COURSE/.venv"; else missing "course environment" "$COURSE/.venv not found: Part 1's lab, tasks 2 and 3"; fi
if [ -f "$COURSE/labbook.md" ]; then ok "lab notebook" "$COURSE/labbook.md"; else missing "lab notebook" "$COURSE/labbook.md not found: Part 1's lab, task 1"; fi

section "6. Hugging Face CLI, login and model storage"
hf_bin=""
if have hf; then hf_bin="$(command -v hf)"; elif [ -x "$COURSE/.venv/bin/hf" ]; then hf_bin="$COURSE/.venv/bin/hf"; fi
if [ -n "$hf_bin" ]; then
  hf_version="$("$hf_bin" version --format json 2>/dev/null | sed -n 's/.*"version": *"\([^"]*\)".*/\1/p')"
  if [ -n "$hf_version" ] && version_ge "$hf_version" "$HF_MIN"; then
    ok "hf" "$hf_version ($hf_bin)"
  else
    missing "hf" "version '${hf_version:-unknown}' is older than $HF_MIN: task 7"
  fi
else
  missing "hf" "not on PATH and not in $COURSE/.venv: task 7"
fi
if [ -d "$MODELS" ]; then ok "model library" "$MODELS"; else missing "model library" "$MODELS not found: Part 4's lab, task 7, or task 7 here"; fi
if [ -f "$MODELS/README.md" ]; then ok "library README" "$MODELS/README.md"; else note "library README" "missing: Part 4's lab, task 7, writes it"; fi
if [ -d "$MODELS/hf" ]; then ok "HF_HOME directory" "$MODELS/hf"; else missing "HF_HOME directory" "$MODELS/hf not found: task 7"; fi
if [ "${HF_HOME:-}" = "$MODELS/hf" ]; then
  ok "HF_HOME" "$HF_HOME"
else
  missing "HF_HOME" "is '${HF_HOME:-unset}' in this shell, expected $MODELS/hf: task 7"
fi
if [ -f "$MODELS/hf/token" ]; then
  ok "stored login" "$MODELS/hf/token"
elif [ -f "$HOME/.cache/huggingface/token" ]; then
  note "stored login" "only under ~/.cache/huggingface, which hf stops reading once HF_HOME moves: task 7"
else
  note "stored login" "none; needed only for gated models (task 7)"
fi
df_target="$MODELS"; [ -d "$df_target" ] || df_target="$HOME"
ok "free disk" "$(df -Pk "$df_target" | awk 'NR == 2 {printf "%d GiB", $4 / 1048576}') on the filesystem holding $df_target"

printf '\nsummary: %d MISSING, %d NOTE\n' "$MISSING" "$NOTES"
[ "$MISSING" -eq 0 ]

Download prepare-mac.sh156 lines

RunnableTrack M · Apple silicon

the starting state, saved to a file

cd ~/llm-course
bash prepare-mac.sh | tee part05-before.txt

Track N — NVIDIA desktop or laptop

RunnableTrack N · NVIDIA GPU

prepare-nvidia.sh

#!/usr/bin/env bash
# Purpose: check an NVIDIA desktop or laptop against everything the rest of the course
#          assumes: the driver and card (and, under WSL2, that no Linux GPU driver was
#          installed inside WSL), the CUDA toolkit, Docker with the NVIDIA runtime and
#          Compose, the toolchain, the Part 1 environment and notebook, the hf CLI and the
#          model storage. It changes nothing.
# Platform: nvidia (Linux, or Windows 11 with WSL2: run it inside the WSL2 Ubuntu shell)
# Minimum memory: 8 GB
# Assumes: the NVIDIA driver on Linux from the distribution, or on Windows from NVIDIA's
#          Windows installer; ~/llm-course from Part 1 (override with COURSE=) and ~/models
#          from Part 4 (override with MODELS=)
#
# Usage: bash prepare-nvidia.sh [--help]
# Output: "ok" (in place), "MISSING" (a later part fails without it; the text names the
#         task that fixes it) or "NOTE" (read it, not blocking), then
#         "summary: N MISSING, M NOTE". Exit status 1 when anything is MISSING.
set -euo pipefail

MODELS="${MODELS:-$HOME/models}"
COURSE="${COURSE:-$HOME/llm-course}"
UV_PIN="0.12.11"
HF_MIN="1.30.0"
INSTALL_HINT="sudo apt install"

for arg in "$@"; do
  case "$arg" in
    -h|--help) sed -n '2,16p' "$0"; exit 0 ;;
    *) echo "prepare-nvidia: unknown argument '$arg' (the script only reports; try --help)" >&2; exit 2 ;;
  esac
done

MISSING=0
NOTES=0
have()    { command -v "$1" >/dev/null 2>&1; }
section() { printf '\n== %s\n' "$1"; }
ok()      { printf '   ok       %-20s %s\n' "$1" "$2"; }
missing() { MISSING=$((MISSING + 1)); printf '   MISSING  %-20s %s\n' "$1" "$2"; }
note()    { NOTES=$((NOTES + 1)); printf '   NOTE     %-20s %s\n' "$1" "$2"; }
# version_ge A B: true when version A >= version B
version_ge() {
  awk -v a="$1" -v b="$2" 'BEGIN {
    na = split(a, x, /[.-]/); nb = split(b, y, /[.-]/); n = (na > nb) ? na : nb
    for (i = 1; i <= n; i++) { xi = (i <= na) ? x[i] + 0 : 0; yi = (i <= nb) ? y[i] + 0 : 0
      if (xi > yi) exit 0; if (xi < yi) exit 1 }
    exit 0 }'
}
first_match() { awk -v pat="$1" '$0 ~ pat && !done { print; done = 1 }'; }

IS_WSL="no"
if grep -qi microsoft /proc/version 2>/dev/null; then IS_WSL="yes"; fi

printf 'prepare-nvidia.sh on %s, %s\n' "$(hostname)" "$(date '+%Y-%m-%d %H:%M')"

section "1. Machine and operating system"
ok "architecture" "$(uname -m)"
kernel="$(uname -r)"
ok "kernel" "$kernel"
if [ "$IS_WSL" = "yes" ]; then
  # The CUDA on WSL guide recommends WSL kernel 5.10.16.3 or later
  if version_ge "$kernel" "5.10.16.3"; then
    ok "WSL2" "yes, kernel at or above the 5.10.16.3 the CUDA on WSL guide recommends"
  else
    missing "WSL2" "kernel $kernel is older than 5.10.16.3: run wsl --update in PowerShell (task 2)"
  fi
else
  ok "WSL2" "no, native Linux"
fi
if [ -r /etc/os-release ]; then
  # shellcheck disable=SC1091
  . /etc/os-release
  ok "distribution" "${PRETTY_NAME:-unknown}"
fi
mem_kib="$(awk '/^MemTotal:/ {print $2}' /proc/meminfo)"
ok "memory (MemTotal)" "$(awk -v k="$mem_kib" 'BEGIN {printf "%.1f GiB of system memory (not VRAM)", k / 1048576}')"

section "2. Driver and card"
smi=""
if have nvidia-smi; then smi="nvidia-smi"; elif [ -x /usr/lib/wsl/lib/nvidia-smi ]; then smi="/usr/lib/wsl/lib/nvidia-smi"; fi
if [ -n "$smi" ]; then
  gpus="$("$smi" --query-gpu=name,driver_version,memory.total --format=csv,noheader 2>/dev/null || true)"
  if [ -n "$gpus" ]; then
    while IFS= read -r gpu; do ok "GPU, driver, VRAM" "$gpu"; done <<< "$gpus"
    driver="$(printf '%s\n' "$gpus" | awk -F', ' 'NR == 1 {print $2}')"
    if ! version_ge "$driver" "580"; then
      note "driver" "$driver is older than 580, the minimum for CUDA 13 (task 2)"
    fi
  else
    missing "GPU" "$smi runs but lists no GPU: see Troubleshooting"
  fi
  [ "$smi" = "nvidia-smi" ] || note "nvidia-smi" "found only at $smi; add /usr/lib/wsl/lib to PATH"
else
  if [ "$IS_WSL" = "yes" ]; then
    missing "nvidia-smi" "not found: install or update the NVIDIA driver on Windows, then wsl --update (task 2)"
  else
    missing "nvidia-smi" "not found: install the driver (task 2)"
  fi
fi
if [ "$IS_WSL" = "yes" ] && have dpkg; then
  linux_drivers="$(dpkg -l 2>/dev/null | awk '$1 == "ii" && ($2 ~ /^nvidia-driver-/ || $2 == "nvidia-open" || $2 ~ /^cuda-drivers/) {printf "%s ", $2}')"
  if [ -n "$linux_drivers" ]; then
    missing "Linux GPU driver" "installed inside WSL (${linux_drivers% }); the CUDA on WSL guide forbids it: Troubleshooting"
  else
    ok "Linux GPU driver" "none inside WSL, as the CUDA on WSL guide requires"
  fi
fi

section "3. CUDA toolkit"
if have nvcc; then
  ok "nvcc" "$(nvcc --version | first_match release)"
elif [ -x /usr/local/cuda/bin/nvcc ]; then
  note "nvcc" "installed at /usr/local/cuda/bin but not on PATH: task 3"
else
  missing "nvcc" "not found; Part 6 compiles llama.cpp with it: task 3"
fi

section "4. Containers"
if have docker; then
  ok "docker" "$(docker --version 2>/dev/null)"
  # shellcheck disable=SC2016  # a Go template for docker, not a shell expansion
  if runtimes="$(docker info --format '{{range $name, $rt := .Runtimes}}{{$name}} {{end}}' 2>/dev/null)"; then
    case " $runtimes " in
      *" nvidia "*) ok "nvidia runtime" "registered (runtimes: ${runtimes% })" ;;
      *)
        if [ "$IS_WSL" = "yes" ]; then
          note "nvidia runtime" "not listed (${runtimes% }); with Docker Desktop, task 6's --gpus test is the check"
        else
          missing "nvidia runtime" "not among the runtimes docker reports (${runtimes% }): task 6"
        fi ;;
    esac
  else
    missing "docker access" "'docker info' failed for $(id -un): daemon stopped, or not in the docker group (task 6)"
  fi
  if compose="$(docker compose version --short 2>/dev/null)"; then
    ok "docker compose" "$compose"
  else
    missing "docker compose" "the Compose plugin is not installed; Part 7 needs it (task 6)"
  fi
else
  missing "docker" "not found: task 6"
fi
if [ "$IS_WSL" = "yes" ]; then
  case "$(cd "$HOME" && pwd -P)/" in
    /mnt/*) missing "home directory" "is on the Windows filesystem; keep the course and models in the WSL2 filesystem" ;;
    *) ok "home directory" "inside the WSL2 filesystem" ;;
  esac
fi

section "5. Toolchain and the Part 1 course directory"
for tool in git python3 curl; do
  if have "$tool"; then
    ok "$tool" "$("$tool" --version 2>&1 | awk 'NR == 1 {print $1, $2, $3}')"
  else
    missing "$tool" "install it with: $INSTALL_HINT $tool"
  fi
done
if have uv; then
  uv_version="$(uv --version | awk '{print $2}')"
  if [ "$uv_version" = "$UV_PIN" ]; then ok "uv" "$uv_version (the course pin)"; else note "uv" "$uv_version; the course is pinned to $UV_PIN (task 5)"; fi
else
  missing "uv" "not found on PATH: task 5"
fi
if [ -x "$COURSE/.venv/bin/python" ]; then ok "course environment" "$COURSE/.venv"; else missing "course environment" "$COURSE/.venv not found: Part 1's lab, tasks 2 and 3"; fi
if [ -f "$COURSE/labbook.md" ]; then ok "lab notebook" "$COURSE/labbook.md"; else missing "lab notebook" "$COURSE/labbook.md not found: Part 1's lab, task 1"; fi

section "6. Hugging Face CLI, login and model storage"
hf_bin=""
if have hf; then hf_bin="$(command -v hf)"; elif [ -x "$COURSE/.venv/bin/hf" ]; then hf_bin="$COURSE/.venv/bin/hf"; fi
if [ -n "$hf_bin" ]; then
  hf_version="$("$hf_bin" version --format json 2>/dev/null | sed -n 's/.*"version": *"\([^"]*\)".*/\1/p')"
  if [ -n "$hf_version" ] && version_ge "$hf_version" "$HF_MIN"; then
    ok "hf" "$hf_version ($hf_bin)"
  else
    missing "hf" "version '${hf_version:-unknown}' is older than $HF_MIN: task 7"
  fi
else
  missing "hf" "not on PATH and not in $COURSE/.venv: task 7"
fi
if [ -d "$MODELS" ]; then ok "model library" "$MODELS"; else missing "model library" "$MODELS not found: Part 4's lab, task 7, or task 7 here"; fi
if [ -f "$MODELS/README.md" ]; then ok "library README" "$MODELS/README.md"; else note "library README" "missing: Part 4's lab, task 7, writes it"; fi
if [ -d "$MODELS/hf" ]; then ok "HF_HOME directory" "$MODELS/hf"; else missing "HF_HOME directory" "$MODELS/hf not found: task 7"; fi
if [ "${HF_HOME:-}" = "$MODELS/hf" ]; then
  ok "HF_HOME" "$HF_HOME"
else
  missing "HF_HOME" "is '${HF_HOME:-unset}' in this shell, expected $MODELS/hf: task 7"
fi
if [ -f "$MODELS/hf/token" ]; then
  ok "stored login" "$MODELS/hf/token"
elif [ -f "$HOME/.cache/huggingface/token" ]; then
  note "stored login" "only under ~/.cache/huggingface, which hf stops reading once HF_HOME moves: task 7"
else
  note "stored login" "none; needed only for gated models (task 7)"
fi
df_target="$MODELS"; [ -d "$df_target" ] || df_target="$HOME"
ok "free disk" "$(df -Pk "$df_target" | awk 'NR == 2 {printf "%d GiB", $4 / 1048576}') on the filesystem holding $df_target"

printf '\nsummary: %d MISSING, %d NOTE\n' "$MISSING" "$NOTES"
[ "$MISSING" -eq 0 ]

Download prepare-nvidia.sh197 lines

RunnableTrack N · NVIDIA GPU

the starting state, saved to a file

cd ~/llm-course
bash prepare-nvidia.sh | tee part05-before.txt

The report has the same shape on every track. This excerpt is from prepare-nvidia.sh on a Linux machine with no NVIDIA driver, toolkit or Docker yet, where Parts 1 and 4 were done, with the host name and values replaced. It omits section 5, whose lines were all ok:

Output — what you should see

prepare-nvidia.sh on <hostname>, 2026-09-13 01:22

== 1. Machine and operating system
   ok       architecture         x86_64
   ok       kernel               6.x.x-xx-generic
   ok       WSL2                 no, native Linux
   ok       distribution         Ubuntu 24.04.x LTS
   ok       memory (MemTotal)    xx.x GiB of system memory (not VRAM)

== 2. Driver and card
   MISSING  nvidia-smi           not found: install the driver (task 2)

== 3. CUDA toolkit
   MISSING  nvcc                 not found; Part 6 compiles llama.cpp with it: task 3

== 4. Containers
   MISSING  docker               not found: task 6
...
== 6. Hugging Face CLI, login and model storage
   ok       hf                   1.30.0 (/home/you/llm-course/.venv/bin/hf)
   ok       model library        /home/you/models
   ok       library README       /home/you/models/README.md
   MISSING  HF_HOME directory    /home/you/models/hf not found: task 7
   MISSING  HF_HOME              is 'unset' in this shell, expected /home/you/models/hf: task 7
   NOTE     stored login         only under ~/.cache/huggingface, which hf stops reading once HF_HOME moves: task 7
   ok       free disk            xxx GiB on the filesystem holding /home/you/models

summary: 5 MISSING, 1 NOTE

Read every MISSING line: together they are your task list. Record nothing by hand yet; part05-before.txt is the record, and task 8 compares it with the end state.

2. Update the operating system and the driver

Track S — NVIDIA DGX Spark

NVIDIA’s update guide “strongly recommend[s]” the DGX Dashboard for every update. It is served at http://localhost:11000 on the Spark; from another machine, the dashboard documentation opens an SSH tunnel first and then browses to the same address:

Fragment — not complete on its own

# on your laptop, not on the Spark
ssh -L 11000:localhost:11000 you@your-spark-hostname

Over SSH without a browser, use the guide’s manual sequence instead.

⚠Warning

sudo apt dist-upgrade may remove packages to resolve dependencies, and fwupdmgr upgrade writes firmware. Read what each proposes before you answer yes, keep the machine on its own power adapter, and never run the firmware steps over a link that might drop. This is the one step in Level 2 where an interruption can leave a machine that does not boot.

RunnableTrack S · DGX Spark

the update guide's manual sequence

sudo apt update
sudo apt dist-upgrade
sudo fwupdmgr refresh
sudo fwupdmgr upgrade
sudo reboot

After the restart, read back what you are now running:

RunnableTrack S · DGX Spark

what the update left

nvidia-smi --query-gpu=name,driver_version --format=csv
uname -r
grep PRETTY_NAME /etc/os-release

Output — what you should see

name, driver_version
NVIDIA GB10, 580.xxx.xx
6.17.x-xxxx-<flavour>
PRETTY_NAME="<distribution and version>"

Compare the driver and kernel with the release notes’ table (580.159.03 and 6.17 on 2026-09-13); a lower driver means the update did not finish. Record: driver version, kernel and PRETTY_NAME, plus the DGX OS version if the dashboard displays one (not checked for this page).

Track X — AMD Ryzen AI Max+ 395

AMD’s Ryzen installation page requires “the 6.14-1018 OEM kernel or newer” for ROCm and gives linux-oem-24.04c as the package. That metapackage follows Ubuntu’s OEM kernel series: on 2026-09-13 it was version 6.17.0-1032.32 in noble-updates, which satisfies the requirement.

RunnableTrack X · Ryzen AI Max+

update Ubuntu, then install the OEM kernel

sudo apt update
sudo apt upgrade
sudo apt install linux-oem-24.04c
sudo reboot

After the restart:

RunnableTrack X · Ryzen AI Max+

confirm which kernel booted

uname -r

Output — what you should see

6.17.0-10xx-oem

The string must end in -oem and be 6.14.0-1018 or later. If it still shows the old -generic kernel, the bootloader chose it: see Troubleshooting. Record the kernel string.

Track M — Apple silicon

Install updates from System Settings > General > Software Update. The command line shows what is pending before you commit to a restart, and reads the version back afterwards:

RunnableTrack M · Apple silicon

pending updates, then the version you are on

softwareupdate --list
sw_vers -productVersion
sw_vers -buildVersion

Output — what you should see

Software Update Tool

Finding available software
No new software available.              (or a list of updates; wording varies by version)
26.x
25xxxx

There is no separate GPU driver on this track: Metal ships inside macOS, so the macOS version and build are the driver version. Record both.

Track N — NVIDIA desktop or laptop

On native Ubuntu (not WSL2), first see whether the driver is already new enough. CUDA 13, which the toolkit in task 3 and Part 1’s cu130 wheels use, needs the 580 series or later.

RunnableTrack N · NVIDIA GPU

is a driver installed, and which

nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv

If that prints a driver of 580 or later, skip to task 3. If nvidia-smi is not found or the driver is older, install the driver Ubuntu recommends with its own tool, which the Ubuntu Server documentation prefers because it installs signed modules that work with Secure Boot.

⚠Warning

Never run this inside WSL2: if grep -qi microsoft /proc/version && echo WSL2 prints WSL2, go to the Windows paragraph below instead. Ubuntu’s documentation warns that if a conflicting NVIDIA driver is installed, ubuntu-drivers “will uninstall your original drivers before installing the new drivers”, and the machine restarts at the end. Save your work first.

RunnableTrack N · NVIDIA GPU

Ubuntu's driver tool

sudo apt update
sudo ubuntu-drivers list
sudo ubuntu-drivers install
sudo reboot

ubuntu-drivers list prints the driver packages available for your card. If the recommended one it installs is older than 580, rerun the install naming a 580-or-later version from that list, for example sudo ubuntu-drivers install nvidia:580, and restart again.

On Windows, install the current NVIDIA driver for your card from NVIDIA’s download page, in Windows. The CUDA on WSL guide is explicit: “This is the only driver you need to install. Do not install any Linux display driver in WSL.” Then, in an administrator PowerShell, install WSL2 if this machine has never had it, and restart Windows when it asks:

RunnableTrack N · Windows

first time only: install WSL2 with Ubuntu

wsl --install

Every time, bring the WSL kernel up to date and confirm the default distribution runs on WSL 2:

RunnableTrack N · Windows

update WSL and read its status

wsl --update
wsl --status

Output — what you should see

Default Distribution: Ubuntu
Default Version: 2               (layout varies by WSL version)

Open the Ubuntu terminal and run the nvidia-smi block above inside it: the Windows driver is mapped into WSL2, so the card appears with no Linux driver installed. Record: card, driver version, memory.total, and on Windows the output of wsl --version.

3. Install the compute stack for your track

Track S — NVIDIA DGX Spark

Nothing to install: DGX OS ships the CUDA toolkit. What is often missing is nvcc on your PATH, and Part 6’s build needs it. The CUDA installation guide describes /usr/local/cuda as a symbolic link to the installed toolkit, so a profile line naming it survives toolkit updates. The grep makes the command safe to run twice.

RunnableTrack S · DGX Spark

put the toolkit on PATH, once

grep -qs '/usr/local/cuda/bin' ~/.bashrc || echo 'export PATH="/usr/local/cuda/bin:$PATH"' >> ~/.bashrc
export PATH="/usr/local/cuda/bin:$PATH"
nvcc --version

Output — what you should see

nvcc: NVIDIA (R) Cuda compiler driver
...
Cuda compilation tools, release 13.0, V13.0.xx

The release line must say 13.0 or later. Record it.

Track X — AMD Ryzen AI Max+ 395

Two GPU paths, installed in this order. Vulkan is what llama.cpp uses on this track from Part 6 and is the course default; ROCm is what PyTorch needs from Part 11 onwards.

Vulkan. Mesa’s Vulkan driver for AMD GPUs is RADV, and vulkaninfo is the check the Vulkan SDK’s own guide uses:

RunnableTrack X · Ryzen AI Max+

Mesa's Vulkan driver and the tools to query it

sudo apt install mesa-vulkan-drivers vulkan-tools
vulkaninfo --summary

Output — what you should see

Devices:
========
GPU0:
        apiVersion         = 1.x.xxx
        deviceType         = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
        deviceName         = <a name containing Radeon or RADV>
        driverName         = <radv>
GPU1:
        deviceName         = llvmpipe (...)

Look for a device of type INTEGRATED_GPU whose name mentions Radeon or RADV. A device named llvmpipe is Mesa’s software renderer on the CPU; if it is the only device listed, see Troubleshooting. The exact names were not captured on hardware for this page.

Which ROCm. AMD’s documentation split on the day this was written, and the choice is decided by the PyTorch wheel, not by which page is newer:

ROCm	Where it is documented (2026-09-13)	Matches Part 1’s torch 2.14.0+rocm7.2	Use it when
7.2.1	AMD’s Ryzen installation page, “ROCm on Radeon and Ryzen”, which covers releases through 7.2.1	yes: torch.version.hip reports 7.2	the course path
10.0.0	the unified ROCm documentation, whose install page says to uninstall 7.2.4 or older first	no	not in this course while Part 1’s wheel is built for 7.2
none	nothing to install	PyTorch runs on the CPU	only if this machine will never train; llama.cpp’s Vulkan path is unaffected

ROCm 7.2.1. The commands are AMD’s Ryzen page verbatim. The --no-dkms flag matters: the page says it “must be passed, as inbox drivers are required for ROCm on Ryzen”, meaning the amdgpu driver already inside the OEM kernel, not AMD’s separately built module.

RunnableTrack X · Ryzen AI Max+

ROCm 7.2.1 for Ryzen, from AMD's installation page

cd ~/llm-course
sudo apt update
wget https://repo.radeon.com/amdgpu-install/7.2.1/ubuntu/noble/amdgpu-install_7.2.1.70201-1_all.deb
sudo apt install ./amdgpu-install_7.2.1.70201-1_all.deb
amdgpu-install -y --usecase=rocm --no-dkms
sudo usermod -a -G render,video $LOGNAME

Do not restart yet: task 4 changes a kernel setting that needs a restart too, and one restart covers both. After that restart, confirm the runtime sees the GPU and your groups took:

RunnableTrack X · Ryzen AI Max+

after the restart in task 4

groups
rocminfo | grep -E "^\s+Name:\s+gfx"
amd-smi version

Output — what you should see

you adm cdrom sudo dip video plugdev render ...
  Name:                    gfx1151
AMDSMI Tool: xx.x.x | AMDSMI Library version: xx.x.x | ROCm version: 7.2.1

groups must contain both video and render; gfx1151 is the name AMD’s compatibility documentation uses for this GPU. Record the amd-smi version line.

Track M — Apple silicon

Metal needs nothing installed. What the later parts need are the Xcode command-line tools, which Homebrew requires for a supported installation, and Homebrew itself, which Part 6 uses to install cmake.

RunnableTrack M · Apple silicon

the command-line tools

xcode-select --install
xcode-select -p

Output — what you should see

/Library/Developer/CommandLineTools

--install opens a dialogue; accept it and wait for it to finish before the second command. If the tools are already installed it says so instead. Any path from -p passes.

Homebrew’s home page says its install script “explains what it will do and then pauses before it does it”; on Apple silicon it installs under /opt/homebrew. The installation documentation then asks you to add brew shellenv to your shell’s configuration file; the grep keeps that to one line however many times you run the block.

RunnableTrack M · Apple silicon

Homebrew, and its line in ~/.zshrc

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
grep -qs 'brew shellenv' ~/.zshrc || echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zshrc
eval "$(/opt/homebrew/bin/brew shellenv)"
brew --version

Output — what you should see

Homebrew x.y.z

Record the Homebrew version.

Track N — NVIDIA desktop or laptop

The driver runs CUDA programs; the CUDA toolkit compiles them, and Part 6 compiles llama.cpp. Install it from NVIDIA’s network repository. The repository name differs between native Ubuntu and WSL2, so the first line sets it once and the rest use it. The WSL guide recommends the WSL-Ubuntu repository because “the default CUDA Toolkit comes packaged with a driver”; the CUDA 13.4 installation guide states that none of its toolkit metapackages depend on the driver any more. The two NVIDIA pages disagree by date, and inside WSL2 this page follows the WSL guide.

RunnableTrack N · NVIDIA GPU

the CUDA 13 toolkit from NVIDIA's repository

cd ~/llm-course
if grep -qi microsoft /proc/version; then REPO=wsl-ubuntu; else REPO="ubuntu$(. /etc/os-release && echo "$VERSION_ID" | tr -d .)"; fi
echo "$REPO"
wget "https://developer.download.nvidia.com/compute/cuda/repos/$REPO/x86_64/cuda-keyring_1.1-1_all.deb"
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda-toolkit-13

echo prints ubuntu2404 on Ubuntu 24.04 and wsl-ubuntu inside WSL2. cuda-toolkit-13 installs the newest 13.x toolkit and, per the guide’s metapackage table, “Will not upgrade beyond the 13.x series toolkits”; apt’s Need to get line should be in the region of the 3.44 GB in the Requirements table. Then put it on PATH, once:

RunnableTrack N · NVIDIA GPU

put the toolkit on PATH, once

grep -qs '/usr/local/cuda/bin' ~/.bashrc || echo 'export PATH="/usr/local/cuda/bin:$PATH"' >> ~/.bashrc
export PATH="/usr/local/cuda/bin:$PATH"
nvcc --version

Output — what you should see

nvcc: NVIDIA (R) Cuda compiler driver
...
Cuda compilation tools, release 13.x, V13.x.xx

Record the release line.

4. See the GPU, and decide how much memory it may use

A GPU that is listed is not yet a GPU that can hold your model. On Tracks S and N the ceiling is fixed by hardware and there is nothing to set. On Tracks X and M a software limit sits below the machine’s total, and the defaults are conservative.

The mechanism on Track X. The amdgpu driver lets the GPU map system memory through its GTT (graphics translation table) domain. In the kernel sources for both 6.14 and 6.17, the GTT size is the TTM memory manager’s page limit multiplied by the page size, and that limit defaults to half of system memory. AMD’s Ryzen page describes the same thing from the outside: ROCm’s shared pool “is configured by default to half the system memory”, and can be raised through /sys/module/ttm/parameters/pages_limit. On Track M, macOS caps how much memory may be wired for the GPU with the iogpu.wired_limit_mb sysctl; MLX’s documentation calls the cap “the system wired limit” and reports it as max_recommended_working_set_size.

Pseudocode — not a real command

default TTM limit (Track X)   = MemTotal / 2
GTT size                      = pages_limit × 4096 bytes
pages for L GiB               = L × 1024³ / 4096 = L × 262,144
wired_limit_mb for L GiB      = L × 1024
course limit L, in whole GiB  = floor( min( total − 10 GB,  0.90 × total ) )
if L ≤ total / 2              set nothing: the default already allows at least as much

The course limit keeps back Part 4’s 10 GB reserve for the operating system, and never goes above 90 per cent of the total, the threshold above which amd-ttm stops to ask whether you are sure. The same rule applies on both unified-memory tracks. Here it is on AMD’s own example machine, whose amd-ttm output shows 125.65 GiB of total memory and a limit of 16,469,033 pages, with a 16 GiB machine in the last row, where the reserve leaves less than the default already gives:

Quantity	Arithmetic	Result
Default limit	16,469,033 pages × 4,096 bytes	67,457,159,168 bytes = 62.82 GiB, half of 125.65
What the kernel logs for it	67,457,159,168 ÷ 1,048,576	amdgpu: 64332M of GTT memory ready.
AMD’s example, amd-ttm --set 100	100 × 262,144	26,214,400 pages = 100.00 GiB
The 90 per cent prompt	0.90 × 125.65	113.09 GiB
Course limit	floor(min(125.65 − 9.31, 113.09))	113 GiB = 29,622,272 pages; 115,712 MB as a wired limit
A 16 GiB machine	16 − 9.31 = 6.69 GiB after the reserve; floor gives 6, not above 16 ÷ 2 = 8	nothing is set: half of memory, the Track X default, is already 8 GiB

This snippet does the same arithmetic for your machine. It reads the total from /proc/meminfo or sysctl hw.memsize, or takes a total in GiB as an argument so you can check someone else’s numbers. Save it as gpu-limit.py in ~/llm-course; it needs only the standard library.

RunnableAll tracks

gpu-limit.py

import platform, subprocess, sys

GIB = 1024 ** 3
PAGE = 4096                    # bytes per TTM page
RESERVE = 10e9 / GIB           # Part 4's 10 GB reserve, expressed in GiB

if len(sys.argv) > 1:          # a total in GiB, to redo someone else's arithmetic
    total = float(sys.argv[1]) * GIB
elif platform.system() == "Darwin":
    total = int(subprocess.run(["sysctl", "-n", "hw.memsize"],
                               capture_output=True, text=True).stdout)
else:
    with open("/proc/meminfo") as f:
        total = next(int(l.split()[1]) * 1024 for l in f if l.startswith("MemTotal:"))

t = total / GIB
limit = int(min(t - RESERVE, 0.90 * t))   # whole GiB, below both ceilings
print(f"total memory                 {t:8.2f} GiB")
print(f"kernel default TTM limit     {t / 2:8.2f} GiB = {int(total / 2 / PAGE):,} pages")
print(f"amd-ttm asks to confirm over {0.90 * t:8.2f} GiB (90%)")
print(f"total minus 10 GB reserve    {t - RESERVE:8.2f} GiB")
if limit <= t / 2:             # at or below the default: raising it would lower it
    print(f"course limit                 none: not above the {t / 2:.2f} GiB default")
    print("  Track X: leave the TTM limit unchanged")
    print("  Track M: leave iogpu.wired_limit_mb at 0")
    sys.exit(0)
print(f"course limit                 {limit:8d} GiB")
print(f"  Track X: amd-ttm --set {limit}   ({limit * GIB // PAGE:,} pages)")
print(f"  Track M: sudo sysctl iogpu.wired_limit_mb={limit * 1024}")
print(f"           only if max_recommended_working_set_size / 1024**3 is below {limit}")

Output — what you should see

$ python3 gpu-limit.py 125.65
total memory                   125.65 GiB
kernel default TTM limit        62.83 GiB = 16,469,196 pages
amd-ttm asks to confirm over   113.09 GiB (90%)
total minus 10 GB reserve      116.34 GiB
course limit                      113 GiB
  Track X: amd-ttm --set 113   (29,622,272 pages)
  Track M: sudo sysctl iogpu.wired_limit_mb=115712
           only if max_recommended_working_set_size / 1024**3 is below 113

$ python3 gpu-limit.py 64
...
course limit                       54 GiB
  Track X: amd-ttm --set 54   (14,155,776 pages)
  Track M: sudo sysctl iogpu.wired_limit_mb=55296
           only if max_recommended_working_set_size / 1024**3 is below 54

$ python3 gpu-limit.py 16
total memory                    16.00 GiB
kernel default TTM limit         8.00 GiB = 2,097,152 pages
amd-ttm asks to confirm over    14.40 GiB (90%)
total minus 10 GB reserve        6.69 GiB
course limit                 none: not above the 8.00 GiB default
  Track X: leave the TTM limit unchanged
  Track M: leave iogpu.wired_limit_mb at 0

The default line differs from AMD’s 16,469,033 pages in the last digits only because 125.65 is AMD’s rounding of the real total. On your machine, run it without an argument and keep the output for the notebook:

RunnableAll tracks

the arithmetic for this machine

cd ~/llm-course
python3 gpu-limit.py

Track S — NVIDIA DGX Spark

RunnableTrack S · DGX Spark

the GPU, and the memory it shares

nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
grep MemTotal /proc/meminfo

Output — what you should see

name, driver_version, memory.total [MiB]
NVIDIA GB10, 580.xxx.xx, <a figure, or [N/A]>
MemTotal:       xxxxxxxxx kB

On this unified-memory machine the GPU’s memory is system memory, so MemTotal is the pool; the memory.total column was not captured on hardware for this page and may be empty. NVIDIA’s known-issues page adds that a plain nvidia-smi shows Memory-Usage: Not Supported, and that cudaMemGetInfo can report less than is really allocatable. Nothing to set. Record the name, the driver and MemTotal converted to GiB (kB ÷ 1,048,576).

Track X — AMD Ryzen AI Max+ 395

First read what the driver is offering now. grep -H prints each file with its value, whatever the card is numbered.

RunnableTrack X · Ryzen AI Max+

VRAM, GTT and the TTM limit, before

grep -H . /sys/class/drm/card*/device/mem_info_vram_total /sys/class/drm/card*/device/mem_info_gtt_total
cat /sys/module/ttm/parameters/pages_limit
sudo dmesg | grep -E "M of (VRAM|GTT) memory ready"

Output — what you should see

/sys/class/drm/card1/device/mem_info_vram_total:xxxxxxxxx
/sys/class/drm/card1/device/mem_info_gtt_total:xxxxxxxxxxx
xxxxxxxx
[    x.xxxxxx] amdgpu 0000:xx:00.0: amdgpu: xxxM of VRAM memory ready
[    x.xxxxxx] amdgpu 0000:xx:00.0: amdgpu: xxxxxM of GTT memory ready.

The two sysfs files are in bytes, the limit is in pages, and the log lines are in MiB (the prefix before amdgpu: differs between kernel versions). Check them against each other: mem_info_gtt_total should equal pages_limit × 4096, and roughly half of MemTotal on a fresh install. AMD also recommends setting the firmware’s dedicated VRAM to its minimum, 0.5 GB, and raising the TTM limit instead; the check script prints a NOTE when the carve-out is larger. The firmware menu that sets it differs by manufacturer.

Now install AMD’s helper, as its Ryzen page does:

RunnableTrack X · Ryzen AI Max+

amd-ttm, from AMD's amd-debug-tools

sudo apt install pipx
pipx ensurepath
pipx install "amd-debug-tools==0.2.21"

pipx ensurepath adds pipx’s directory to your shell profile, so open a new terminal, then confirm the version and query the current setting:

RunnableTrack X · Ryzen AI Max+

in the new terminal: the version, and what amd-ttm sees

amd-ttm --version
amd-ttm

Output — what you should see

0.2.21
💻 Current TTM pages limit: xxxxxxxx pages (xx.xx GB)
💻 Total system memory: xxx.xx GB

Despite the label, amd-ttm converts with 1024³ bytes per “GB”, so its figures are GiB and match gpu-limit.py. amd-ttm --set re-runs itself with sudo, writes one line, options ttm pages_limit=<pages>, to /etc/modprobe.d/ttm.conf, regenerates the initramfs if the ttm module is inside it, and offers to restart. Answer y: this is also the restart ROCm’s group change in task 3 waits for.

⚠Warning

This writes a kernel module option that applies at every boot, rebuilds the initramfs when needed, and restarts the machine. Use the number gpu-limit.py printed for your machine, not the 113 from AMD’s example machine, and save your work before answering y. If it printed none, do not run amd-ttm --set; run sudo reboot instead, for task 3’s group change.

RunnableTrack X · Ryzen AI Max+

set the limit (use your own number), then restart

amd-ttm --set 113

The first two lines and the prompt are the format AMD’s page shows; the initramfs line is printed by version 0.2.21, read from its source rather than run on hardware.

Output — what you should see

🐧 Successfully set TTM pages limit to 29622272 pages (113.00 GB)
🐧 Configuration written to /etc/modprobe.d/ttm.conf
🐧 Checking if the initramfs image needs to be regenerated
...                                  (initramfs lines; they depend on your system)
○ NOTE: You need to reboot for changes to take effect.
Would you like to reboot the system now? (y/n): y

After the restart, run the “before” block again. pages_limit must equal the pages amd-ttm printed, and the GTT log line must show the new size in MiB: 113 GiB is 115712M. Then run the ROCm check at the end of task 3. Record: VRAM carve-out, the old and new pages_limit, the new mem_info_gtt_total in GiB, and the amd-ttm --set value.

Track M — Apple silicon

Read the three numbers MLX’s documentation tells you to compare: the total, the system wired cap, and the current sysctl.

RunnableTrack M · Apple silicon

total memory, the wired cap, and the sysctl

cd ~/llm-course
source .venv/bin/activate
system_profiler SPDisplaysDataType | grep -E "Chipset Model|Total Number of Cores"
sysctl -n hw.memsize
sysctl -n iogpu.wired_limit_mb
python -c "import mlx.core as mx; i = mx.device_info(); print(i['memory_size'], i['max_recommended_working_set_size'])"

Output — what you should see

      Chipset Model: Apple M4 Pro
      Total Number of Cores: xx
xxxxxxxxxxx
0
xxxxxxxxxxx xxxxxxxxxxx

hw.memsize and memory_size are bytes; 0 from the sysctl means the system default is in force. Take the course limit gpu-limit.py printed, then apply the decision rule: if it printed none, or if max_recommended_working_set_size, divided by 1024³, is already at or above the course limit, leave the sysctl at 0, because setting a lower number would shrink what the GPU may hold. Otherwise set it:

⚠Warning

This changes how much memory the whole system will pin for the GPU. Set it too close to the total and macOS is left with too little for everything else, which shows up as a frozen desktop rather than an error. Close large applications first. The Apple silicon lesson notes that the setting does not survive a restart, so the check script prints it each time and later labs tell you when to apply it again. Use the number gpu-limit.py printed, not the 64 GB example below.

RunnableTrack M · Apple silicon

raise the wired limit (use your own number)

sudo sysctl iogpu.wired_limit_mb=55296
sysctl -n iogpu.wired_limit_mb
python -c "import mlx.core as mx; print(mx.device_info()['max_recommended_working_set_size'])"

Output — what you should see

iogpu.wired_limit_mb: 0 -> 55296       (layout may differ by macOS version)
55296
xxxxxxxxxxx

The second line must echo your number. Whether MLX’s reported cap moves with the sysctl was not verified for this page: record what it prints either way. Record: chip and core count, hw.memsize, the cap before and after, and the sysctl value you set.

Track N — NVIDIA desktop or laptop

RunnableTrack N · NVIDIA GPU

the card and its VRAM

nvidia-smi --query-gpu=name,driver_version,memory.total,memory.used --format=csv

Output — what you should see

name, driver_version, memory.total [MiB], memory.used [MiB]
NVIDIA GeForce RTX xxxx, 5xx.xx.xx, xxxxx MiB, xxx MiB

The ceiling here is the card’s VRAM and nothing configures it. memory.total is in MiB: divide by 1,024 for GiB, and use that figure, not system memory, in Part 4’s fit arithmetic. memory.used above a few hundred MiB with nothing running means a desktop session or another program holds part of the card; note it, because it comes off every budget. If nvidia-smi is not found inside WSL2, see Troubleshooting. Record: card, driver, memory.total, memory.used at idle.

5. Git, Python and uv

Part 1 installed these; here you confirm them against the pins and set one thing Part 25 needs: Aider commits your changes to Git by default, and a commit needs a Git identity.

RunnableAll tracks

versions, and the Git identity

git --version
~/llm-course/.venv/bin/python --version
uv --version
command -v uv
git config --global user.name
git config --global user.email

Output — what you should see

git version 2.xx.x
Python 3.12.x
uv 0.12.11 (<arch>-<os>)
/home/you/.local/bin/uv
Your Name
you@example.com

If the two git config lines print nothing, set them with git config --global user.name "Your Name" and git config --global user.email "you@example.com". If uv --version shows anything other than 0.12.11, the command that pins it depends on how uv was installed, which command -v uv tells you:

command -v uv prints	Installed by	Pin it with
~/.local/bin/uv	the standalone installer (Part 1)	uv self update 0.12.11
/opt/homebrew/bin/uv	Homebrew	brew uninstall uv, then curl -LsSf https://astral.sh/uv/0.12.11/install.sh | sh and a new terminal
nothing	not installed	curl -LsSf https://astral.sh/uv/0.12.11/install.sh | sh, then a new terminal

uv self update takes a target version as its argument, and uv’s installation page documents the version-in-the-URL form of the installer; self-update is disabled for any other installation method, which is why Homebrew’s copy has to go. After pinning, run uv --version again (in a new terminal where the table says so); it must print uv 0.12.11. Record: Git version and identity, the Python and uv versions.

✦Tip

If your Part 1 environment is in a state you no longer understand, deleting .venv and rebuilding it with Part 1’s setup-env.sh takes minutes, and is faster than debugging it.

6. Containers, where your track uses them

Part 7’s private chat service runs its engine and front end under Docker Engine with the Compose plugin on Tracks S, X and N, with the NVIDIA runtime on S and N and AMD’s device files on X.

Track S — NVIDIA DGX Spark

The user guide states that “The NVIDIA Container Toolkit is preinstalled and configured on DGX Spark systems”. Adding yourself to the docker group is optional there; without it, prefix every docker command with sudo.

⛨Security

Docker’s post-installation page is blunt: “The docker group grants root-level privileges to the user.” Add only your own account, and only on a machine you administer.

RunnableTrack S · DGX Spark

join the docker group (skip if you prefer sudo)

sudo usermod -aG docker $USER
newgrp docker

A group change otherwise reaches only new logins, so newgrp starts a new shell with the group active; run the next block in it.

RunnableTrack S · DGX Spark

runtime, Compose, and a GPU inside a container

docker info --format '{{range $name, $rt := .Runtimes}}{{$name}} {{end}}'
docker compose version
docker run --rm --gpus=all nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04 nvidia-smi

Output — what you should see

io.containerd.runc.v2 nvidia runc
Docker Compose version vx.x.x
Unable to find image 'nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04' locally
...
| NVIDIA-SMI 580.xxx.xx      Driver Version: 580.xxx.xx      CUDA Version: 13.x |
...
|   0  NVIDIA GB10 ...

The runtime line is inferred from the toolkit’s --runtime=nvidia sample, not captured on a Spark. The pass is a Compose version and the GB10 in the table from inside the container, the user guide’s own test; its 3.9 GB compressed image is reused later. If nvidia is not among the runtimes, the check script prints a NOTE, but Part 7’s preflight passes --runtime=nvidia: apply the runtime fix in Troubleshooting first.

If docker compose version fails, the Compose plugin is not installed. This installs it under the package name from Docker’s Ubuntu instructions, not checked against DGX OS:

RunnableTrack S · DGX Spark

only if docker compose version failed: the Compose plugin

sudo apt update
sudo apt install docker-compose-plugin
docker compose version

Output — what you should see

Docker Compose version vx.x.x

If you joined the docker group, log out and back in, or restart, before task 7: a new SSH login gets the group, a new terminal window in the same desktop session does not. Confirm:

RunnableTrack S · DGX Spark

after logging back in: the group in a new shell

id -nG | tr ' ' '\n' | grep -x docker

Output — what you should see

docker

Record: Docker and Compose versions, whether nvidia is listed among the runtimes, and whether you use the group or sudo.

Track X — AMD Ryzen AI Max+ 395

Install Docker Engine from Docker’s own repository, as its Ubuntu page documents. That page first removes “unofficial Docker packages” from the distribution, which may conflict; check whether you have any with dpkg -l docker.io podman-docker containerd runc.

⚠Warning

Only if that dpkg -l shows lines starting ii: Docker’s page removes the conflicting packages with the command below. It uninstalls those packages, and containers and images created with them stop being managed by the new engine. Read the list apt prints before confirming.

RunnableTrack X · Ryzen AI Max+

remove conflicting packages, only if present

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-compose-v2 docker-doc docker-buildx podman-docker containerd runc | cut -f1)

RunnableTrack X · Ryzen AI Max+

Docker Engine and Compose from Docker's apt repository

sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Then the group (see the security note in the Track S tab):

RunnableTrack X · Ryzen AI Max+

use Docker without sudo

sudo usermod -aG docker $USER
newgrp docker

newgrp starts a new shell with the group active; run the test in it:

RunnableTrack X · Ryzen AI Max+

prove the engine and Compose work

docker run --rm hello-world
docker compose version

Output — what you should see

...
Hello from Docker!
This message shows that your installation appears to be working correctly.
...
Docker Compose version vx.x.x

No GPU is needed inside a container yet. Part 7 passes /dev/kfd and /dev/dri into its ROCm container; the check script confirms /dev/kfd exists. Podman is accepted where a lab says so, but Part 7’s lab is written for Docker Engine.

Now log out of your desktop session and back in, or restart, before task 7: newgrp gave the docker group to one shell, and a new terminal window does not get it. Confirm:

RunnableTrack X · Ryzen AI Max+

after logging back in: the group in a new shell

id -nG | tr ' ' '\n' | grep -x docker

Output — what you should see

docker

Record: Docker and Compose versions.

Track M — Apple siliconPartial

Linux containers on macOS run in a virtual machine with no access to the Mac's GPU, so the course's container labs take a native path on Track M.

Nothing to install. Part 7’s lab runs Ollama natively, its front end from Python and its proxy from Homebrew. Docker Desktop or Podman installed for other work can stay.

Track N — NVIDIA desktop or laptop

On native Ubuntu (not WSL2), install Docker Engine from Docker’s repository, then the NVIDIA Container Toolkit, then register its runtime with Docker. Docker’s Ubuntu page first removes “unofficial Docker packages” that may conflict; check whether you have any with dpkg -l docker.io podman-docker containerd runc.

⚠Warning

Only if that dpkg -l shows lines starting ii: Docker’s page removes the conflicting packages with the command below. It uninstalls those packages, and containers and images created with them stop being managed by the new engine. Read the list apt prints before confirming.

RunnableTrack N · NVIDIA GPU

remove conflicting packages, only if present

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-compose-v2 docker-doc docker-buildx podman-docker containerd runc | cut -f1)

Then the engine, exactly as Docker’s Ubuntu page gives it:

RunnableTrack N · NVIDIA GPU

Docker Engine and Compose from Docker's apt repository

sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Then the group (see the security note in the Track S tab):

RunnableTrack N · NVIDIA GPU

use Docker without sudo

sudo usermod -aG docker $USER
newgrp docker

newgrp starts a new shell with the group active; run the rest of this path in it. First the toolkit, from NVIDIA’s installation guide, at the version it named on 2026-09-13:

RunnableTrack N · NVIDIA GPU

the NVIDIA Container Toolkit, registered with Docker

sudo apt-get update && sudo apt-get install -y --no-install-recommends ca-certificates curl gnupg2
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.20.0-1
sudo apt-get install -y \
    nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

nvidia-ctk runtime configure edits /etc/docker/daemon.json so Docker knows a runtime named nvidia. Prove it with the guide’s sample workload, which runs nvidia-smi inside a plain Ubuntu image (41.6 MB compressed for amd64 on Docker Hub):

RunnableTrack N · NVIDIA GPU

runtime, Compose, and a GPU inside a container (Linux)

docker info --format '{{range $name, $rt := .Runtimes}}{{$name}} {{end}}'
docker compose version
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Output — what you should see

io.containerd.runc.v2 nvidia runc
Docker Compose version vx.x.x
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 5xx.xx.xx              Driver Version: 5xx.xx.xx      CUDA Version: 13.x     |
...
|   0  NVIDIA GeForce RTX xxxx ...

Then log out of your desktop session and back in, or restart, before task 7: newgrp gave the docker group to one shell, and a new terminal window does not get it. Confirm:

RunnableTrack N · NVIDIA GPU

after logging back in: the group in a new shell

id -nG | tr ' ' '\n' | grep -x docker

Output — what you should see

docker

On Windows with WSL2, use Docker Desktop instead: its documentation states that GPU support in Docker Desktop “is only available on Windows with the WSL2 backend”. Install it in Windows, keep the WSL 2 engine selected, and turn on integration for Ubuntu under Settings > Resources > WSL Integration. Then, in the Ubuntu terminal, run Docker’s own validation command:

RunnableTrack N · NVIDIA GPU

a GPU inside a container (WSL2 with Docker Desktop)

docker compose version
docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

Output — what you should see

Docker Compose version vx.x.x
...
> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
...
GPU Device 0: "<your card's name>" with compute capability x.x

> Compute x.x CUDA device: [<your card's name>]
xxxxx bodies, total time for 10 iterations: xx.xxx ms
= xxx.xxx billion interactions per second
= xxxx.xxx single-precision GFLOP/s at 20 flops per interaction

The shape is from Docker’s GPU support page, abridged: the sample first prints its options and a note, and the wording may differ by sample version. The pass is the GPU Device 0: line naming your card. Under Docker Desktop the runtime list may not name nvidia, so the check script reports a NOTE rather than MISSING, and this test is the check. Record: Docker, Compose and toolkit versions, and which of the two paths you took.

7. The Hugging Face CLI, a login, and one place for models

This is the library Part 4’s shortlist lab created: ~/models/ holds your verified GGUF files, its README.md reserves ~/models/hf, and this task puts the Hugging Face home there. Engines that take a file path read the GGUF tree; transformers, vLLM, SGLang and mlx-lm download through the Hugging Face cache, which after this task sits on the same disk. Every command below is safe to run on a machine where Part 4 already did some of the work.

The CLI. It comes with huggingface_hub in the course environment. The version specifier makes the install a no-op when Part 4 already upgraded it:

RunnableAll tracks

the hf CLI in the course environment, at 1.30.0 or later

cd ~/llm-course
source .venv/bin/activate
uv pip install "huggingface_hub>=1.30.0"
hf version

Output — what you should see

Using Python 3.12.x environment at: /home/you/llm-course/.venv
Checked 1 package in xxms
✓ hf version
  version: 1.30.x

Checked 1 package means nothing needed installing; an older version prints the packages uv replaced instead.

The storage. What moving the Hugging Face home involves, one level down: the library reads HF_HOME when it is imported and derives HF_HUB_CACHE as $HF_HOME/hub and HF_TOKEN_PATH as $HF_HOME/token, with stored_tokens beside it. A browser login from Part 2 or Part 4 therefore lives at ~/.cache/huggingface/token, and hf stops seeing it the moment HF_HOME changes. The cache itself can be moved: version 1.30.0 creates the links inside it as relative symbolic links, which its source notes survive the cache folder being moved. The script below does this idempotently: it adds only what is missing, copies the login with mode 600 (the mode hf itself uses), never overwrites or deletes anything, and moves the old cache only when asked.

RunnableAll tracks

setup-model-storage.sh

#!/usr/bin/env bash
# Purpose: make ~/models/hf the Hugging Face home (HF_HOME) on this machine, idempotently:
#          create it inside the model library Part 4 created, add HF_HOME to your shell
#          profile once, copy an existing Hugging Face login across, and optionally move
#          an existing hub cache into it
# Platform: all (Linux, WSL2 and macOS; bash or zsh as the login shell)
# Minimum memory: 8 GB
# Assumes: Part 4's library at ~/models (created here if it is missing); SHELL names bash
#          or zsh; nothing is ever deleted, and nothing already in place is overwritten
#
# Usage: bash setup-model-storage.sh [--check] [--move-cache]
#        --check       report what would change, and change nothing
#        --move-cache  also move an existing ~/.cache/huggingface/hub to ~/models/hf/hub,
#                      only if ~/models/hf/hub does not exist yet
#        --help        print this header
#
# Running it twice is safe: the second run reports "ok" for everything the first created.
set -euo pipefail

MODELS="${MODELS:-$HOME/models}"
NEW_HOME="$MODELS/hf"
OLD_HOME="${XDG_CACHE_HOME:-$HOME/.cache}/huggingface"
CHECK="no"
MOVE_CACHE="no"

for arg in "$@"; do
  case "$arg" in
    --check) CHECK="yes" ;;
    --move-cache) MOVE_CACHE="yes" ;;
    -h|--help) sed -n '2,17p' "$0"; exit 0 ;;
    *) echo "setup-model-storage: unknown argument '$arg' (try --help)" >&2; exit 2 ;;
  esac
done

CHANGED=0
NOTES=0
ok()      { printf '   ok       %-20s %s\n' "$1" "$2"; }
changed() { CHANGED=$((CHANGED + 1)); printf '   %-8s %-20s %s\n' "$ACTION" "$1" "$2"; }
note()    { NOTES=$((NOTES + 1)); printf '   NOTE     %-20s %s\n' "$1" "$2"; }
if [ "$CHECK" = "yes" ]; then ACTION="WOULD"; else ACTION="DONE"; fi

# The line written to the profile. With the default library it stays portable ($HOME).
if [ "$MODELS" = "$HOME/models" ]; then
  # shellcheck disable=SC2016  # the literal $HOME is intended: the profile expands it
  PROFILE_LINE='export HF_HOME="$HOME/models/hf"'
else
  PROFILE_LINE="export HF_HOME=\"$NEW_HOME\""
fi

case "$(basename "${SHELL:-}")" in
  zsh) PROFILE="$HOME/.zshrc" ;;
  bash)
    if [ "$(uname -s)" = "Darwin" ]; then PROFILE="$HOME/.bash_profile"; else PROFILE="$HOME/.bashrc"; fi ;;
  *)
    echo "setup-model-storage: SHELL is '${SHELL:-unset}', not bash or zsh." >&2
    echo "Add this line to your shell's startup file yourself, then rerun with --check:" >&2
    echo "  $PROFILE_LINE" >&2
    exit 1 ;;
esac

printf 'setup-model-storage.sh, mode: %s\n' "$([ "$CHECK" = "yes" ] && echo check || echo apply)"
printf '   library  %s\n   HF_HOME  %s\n   profile  %s\n\n' "$MODELS" "$NEW_HOME" "$PROFILE"

# 1. The directory, inside Part 4's library
if [ -d "$NEW_HOME" ]; then
  ok "directory" "$NEW_HOME exists"
else
  [ "$CHECK" = "yes" ] || mkdir -p "$NEW_HOME"
  changed "directory" "create $NEW_HOME"
fi
if [ -f "$MODELS/README.md" ]; then
  ok "library README" "$MODELS/README.md (Part 4)"
else
  note "library README" "missing: Part 4's lab, task 7, puts it there; the layout still works"
fi

# 2. The profile line, added once
if [ -f "$PROFILE" ] && grep -Eq '^[[:space:]]*export[[:space:]]+HF_HOME=' "$PROFILE"; then
  existing="$(grep -E '^[[:space:]]*export[[:space:]]+HF_HOME=' "$PROFILE" | tail -n 1 | sed 's/^[[:space:]]*//')"
  if [ "$existing" = "$PROFILE_LINE" ]; then
    ok "profile" "$PROFILE already sets HF_HOME"
  else
    note "profile" "$PROFILE already has: $existing (left unchanged; edit it by hand if it is wrong)"
  fi
else
  if [ "$CHECK" = "no" ]; then
    printf '\n# Hugging Face home inside the model library (Local LLM course, Part 5)\n%s\n' \
      "$PROFILE_LINE" >> "$PROFILE"
  fi
  changed "profile" "append to $PROFILE: $PROFILE_LINE"
fi

# 3. An existing login: hf reads the token from $HF_HOME/token, so copy it across
for f in token stored_tokens; do
  if [ -e "$NEW_HOME/$f" ]; then
    ok "login: $f" "present in $NEW_HOME"
  elif [ -f "$OLD_HOME/$f" ]; then
    [ "$CHECK" = "yes" ] || install -m 600 "$OLD_HOME/$f" "$NEW_HOME/$f"
    changed "login: $f" "copy $OLD_HOME/$f (mode 600)"
  else
    ok "login: $f" "none stored under $OLD_HOME, nothing to copy"
  fi
done

# 4. An existing hub cache: reported, and moved only when asked
if [ -d "$OLD_HOME/hub" ]; then
  size="$(du -sh "$OLD_HOME/hub" 2>/dev/null | cut -f1)"
  if [ -e "$NEW_HOME/hub" ]; then
    note "old hub cache" "$OLD_HOME/hub ($size) and $NEW_HOME/hub both exist; not merging them"
  elif [ "$MOVE_CACHE" = "yes" ]; then
    [ "$CHECK" = "yes" ] || mv "$OLD_HOME/hub" "$NEW_HOME/hub"
    changed "old hub cache" "move $OLD_HOME/hub ($size) to $NEW_HOME/hub"
  else
    note "old hub cache" "$OLD_HOME/hub holds $size; rerun with --move-cache to move it"
  fi
else
  ok "old hub cache" "none at $OLD_HOME/hub"
fi

# 5. This shell
if [ "${HF_HOME:-}" = "$NEW_HOME" ]; then
  ok "this shell" "HF_HOME=$HF_HOME"
else
  note "this shell" "HF_HOME is '${HF_HOME:-unset}'; open a new terminal before using hf"
fi

printf '\nsummary: %d %s, %d NOTE\n' "$CHANGED" "$([ "$CHECK" = "yes" ] && echo "would change" || echo changed)" "$NOTES"

Download setup-model-storage.sh127 lines

Download it into ~/llm-course and first look at what it would do. --check changes nothing:

RunnableAll tracks

what would change

cd ~/llm-course
bash setup-model-storage.sh --check

Output — what you should see

setup-model-storage.sh, mode: check
   library  /home/you/models
   HF_HOME  /home/you/models/hf
   profile  /home/you/.bashrc

   WOULD    directory            create /home/you/models/hf
   ok       library README       /home/you/models/README.md (Part 4)
   WOULD    profile              append to /home/you/.bashrc: export HF_HOME="$HOME/models/hf"
   WOULD    login: token         copy /home/you/.cache/huggingface/token (mode 600)
   WOULD    login: stored_tokens copy /home/you/.cache/huggingface/stored_tokens (mode 600)
   NOTE     old hub cache        /home/you/.cache/huggingface/hub holds xxK; rerun with --move-cache to move it
   NOTE     this shell           HF_HOME is 'unset'; open a new terminal before using hf

summary: 4 would change, 2 NOTE

These runs were captured on a test home directory holding a Part 2 login and one small cached file. On a Mac the profile is ~/.zshrc. A NOTE saying your profile “already has” a different HF_HOME line means the script will leave it alone; edit that line by hand.

Now decide about the old cache. If the report has an old hub cache NOTE and you want those downloads in the library, and df -h ~/models shows room for the size it printed, use --move-cache; otherwise leave it off. Decide now: once hf downloads into the new home, the script refuses to merge the two caches. Run one of these two blocks:

RunnableAll tracks

apply, keeping the old cache where it is

bash setup-model-storage.sh

Output — what you should see

setup-model-storage.sh, mode: apply
...
   DONE     login: stored_tokens copy /home/you/.cache/huggingface/stored_tokens (mode 600)
   NOTE     old hub cache        /home/you/.cache/huggingface/hub holds xxK; rerun with --move-cache to move it
   NOTE     this shell           HF_HOME is 'unset'; open a new terminal before using hf

summary: 4 changed, 2 NOTE

RunnableAll tracks

apply and move the old cache

bash setup-model-storage.sh --move-cache

Output — what you should see

setup-model-storage.sh, mode: apply
...
   DONE     login: stored_tokens copy /home/you/.cache/huggingface/stored_tokens (mode 600)
   DONE     old hub cache        move /home/you/.cache/huggingface/hub (xxK) to /home/you/models/hf/hub
   NOTE     this shell           HF_HOME is 'unset'; open a new terminal before using hf

summary: 5 changed, 1 NOTE

Open a new terminal (after the log-out in task 6, on the tracks that have one), then confirm from hf itself:

RunnableAll tracks

in a new terminal: where hf now looks

cd ~/llm-course
source .venv/bin/activate
echo "$HF_HOME"
hf env | grep -E "HF_HUB_CACHE|HF_TOKEN_PATH|Has saved token"

Output — what you should see

/home/you/models/hf
- Has saved token ?: True
- HF_HUB_CACHE: /home/you/models/hf/hub
- HF_TOKEN_PATH: /home/you/models/hf/token

Has saved token ?: False means there was no login to carry across. You need one only for gated repositories; the anonymous rate limit is the other reason Part 4 gives. To sign in, run hf auth login and follow the browser flow Part 4’s task 2 walks through, then check it:

RunnableAll tracks

who hf thinks you are

hf auth whoami

Output — what you should see

✓ Logged in
  user: your-username
  orgs: ...            (only if your account belongs to organisations)

The signed-in lines were read from the hf 1.30.0 source, not captured with a real account. Not signed in, the same command prints Error: Not logged in and exits with status 1.

⛨Security

A Hugging Face token is a credential. Never put one in a script, a committed .env, a Dockerfile or the lab notebook. hf auth login writes it to $HF_HOME/token readable only by you, and the HF_TOKEN environment variable, when set, overrides the stored token for scripted use. If a token leaks, revoke it in your Hub settings and log in again.

Prove the cache lands in the library. Download one small file, the 726-byte config.json of Qwen3-0.6B (Apache-2.0), into the cache rather than a --local-dir:

RunnableAll tracks

one small file, through the cache

hf download Qwen/Qwen3-0.6B config.json
hf cache ls

Output — what you should see

✓ Downloaded
  path: /home/you/models/hf/hub/models--Qwen--Qwen3-0.6B/snapshots/<commit>/config.json
ID                     SIZE LAST_ACCESSED     LAST_MODIFIED     REFS
--------------------- ----- ----------------- ----------------- ----
model/Qwen/Qwen3-0.6B 726.0 a few seconds ago a few seconds ago main

Found 1 repo(s) for a total of 1 revision(s) and 726.0 on disk.

Captured with hf 1.30.0; on 2026-09-13 <commit> was c1899de289a04d12100db370d81485cdf75e47ca. Not signed in, a warning about unauthenticated requests comes first. The path must start with ~/models/hf/hub: a path under ~/.cache/huggingface means this shell has no HF_HOME. When a later part runs a Hugging Face tool inside a container, mount this directory where the container looks, for example -v "$HF_HOME":/root/.cache/huggingface for a container running as root that does not set its own HF_HOME.

8. Record the baseline

Run your track’s check script again into a second file, and compare:

RunnableAll tracks

the end state, and what changed

cd ~/llm-course
source .venv/bin/activate
TRACK=nvidia        # spark, strix, mac or nvidia
bash prepare-$TRACK.sh | tee part05-after.txt
diff part05-before.txt part05-after.txt

Output — what you should see

1c1
< prepare-nvidia.sh on <hostname>, 2026-09-13 01:22
---
> prepare-nvidia.sh on <hostname>, 2026-09-13 02:31
<n>c<n>
<    MISSING  nvidia-smi           not found: install the driver (task 2)
---
>    ok       GPU, driver, VRAM    NVIDIA GeForce RTX xxxx, 5xx.xx.xx, xxxxx MiB
...
<n>c<n>
< summary: 5 MISSING, 1 NOTE
---
> summary: 0 MISSING, 1 NOTE

The last line of part05-after.txt must read summary: 0 MISSING. Any NOTE left over is a decision you have made; say which in the notebook. Then write the machine-readable record. The script probes whatever is present, leaves out what is not, prints the record and appends it to the notebook as one line.

RunnableAll tracks

record-baseline.py

"""Record the machine and its tool versions as one JSON line in the lab notebook.

Purpose: capture the baseline every later measurement in this course is compared
         against: which machine, which operating system and kernel, which accelerator
         and how much memory it may address, which tool versions, and on what date.
Platform: all (probes that do not apply to a track are left out of the record; nothing
          here needs an accelerator to be present, and nothing is changed)
Minimum memory: 8 GB
Assumes: Python 3.9 or newer, ideally the Part 1 environment (so torch and mlx versions
         are recorded); the notebook file already exists (Part 1 created labbook.md).
         Every shell tool it asks is optional and is recorded as absent if missing.

Usage: python record-baseline.py --labbook labbook.md [--track auto] [--note "..."]
       python record-baseline.py --print     (show the record, write nothing)
"""
import argparse
import json
import os
import platform
import shutil
import subprocess
import sys
from datetime import date
from pathlib import Path

TRACKS = ("auto", "spark", "strix", "mac", "nvidia")
GIB = 1024 ** 3
TTM_PAGE_BYTES = 4096


def run(cmd, timeout=30):
    """Run a command; return its trimmed stdout, or None if absent or failing."""
    exe = cmd[0] if os.path.isabs(cmd[0]) else shutil.which(cmd[0])
    if exe is None or not os.path.exists(exe):
        return None
    try:
        out = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout, check=False)
    except (OSError, subprocess.SubprocessError):
        return None
    if out.returncode != 0:
        return None
    return out.stdout.strip() or None


def first_line(text, contains=""):
    for line in (text or "").splitlines():
        if contains in line:
            return line.strip()
    return None


def read_text(path):
    try:
        return Path(path).read_text(encoding="utf-8").strip()
    except OSError:
        return None


def os_release():
    fields = {}
    for line in (read_text("/etc/os-release") or "").splitlines():
        key, _, value = line.partition("=")
        fields[key] = value.strip('"')
    return fields.get("PRETTY_NAME")


def is_wsl():
    return "microsoft" in (read_text("/proc/version") or "").lower()


def nvidia_smi():
    if shutil.which("nvidia-smi"):
        return "nvidia-smi"
    if os.path.exists("/usr/lib/wsl/lib/nvidia-smi"):
        return "/usr/lib/wsl/lib/nvidia-smi"
    return None


def guess_track():
    if sys.platform == "darwin":
        return "mac"
    if nvidia_smi():
        # The GB10 is the only aarch64 machine in this course with an NVIDIA GPU.
        return "spark" if platform.machine() == "aarch64" else "nvidia"
    if list(Path("/sys/class/drm").glob("card*/device/mem_info_gtt_total")):
        return "strix"
    return "unknown"


def memtotal_bytes():
    if sys.platform == "darwin":
        out = run(["sysctl", "-n", "hw.memsize"])
        return int(out) if out and out.isdigit() else None
    for line in (read_text("/proc/meminfo") or "").splitlines():
        if line.startswith("MemTotal:"):
            return int(line.split()[1]) * 1024
    return None


def machine():
    total = memtotal_bytes()
    info = {
        "arch": platform.machine(),
        "system_memory_gib": round(total / GIB, 2) if total else None,
        "kernel": platform.release(),
    }
    if sys.platform == "darwin":
        info["os"] = "macOS " + (run(["sw_vers", "-productVersion"]) or "unknown")
        info["chip"] = run(["sysctl", "-n", "machdep.cpu.brand_string"])
    else:
        info["os"] = os_release()
        info["wsl2"] = is_wsl()
    return info


def accelerator():
    """What the machine says about its accelerator and the memory it may address."""
    info = {}
    smi = nvidia_smi()
    if smi:
        out = run([smi, "--query-gpu=name,driver_version,memory.total", "--format=csv,noheader"])
        if out:
            info["nvidia_smi"] = out.splitlines()
    nvcc = shutil.which("nvcc") or ("/usr/local/cuda/bin/nvcc" if os.path.exists("/usr/local/cuda/bin/nvcc") else None)
    if nvcc:
        info["nvcc"] = first_line(run([nvcc, "--version"]), "release")
    vk = run(["vulkaninfo", "--summary"])
    if vk:
        devices = [ln.split("=", 1)[1].strip() for ln in vk.splitlines() if "deviceName" in ln and "=" in ln]
        drivers = [ln.split("=", 1)[1].strip() for ln in vk.splitlines() if "driverName" in ln and "=" in ln]
        info["vulkan"] = {"devices": devices, "drivers": drivers}
    rocm = run(["rocminfo"])
    if rocm:
        agents = [ln.split()[1] for ln in rocm.splitlines() if ln.split()[:1] == ["Name:"] and len(ln.split()) > 1 and ln.split()[1].startswith("gfx")]
        info["rocminfo_gfx"] = agents
    amd_smi = run(["amd-smi", "version"])
    if amd_smi:
        info["amd_smi_version"] = first_line(amd_smi)
    for card in sorted(Path("/sys/class/drm").glob("card*/device")):
        vram, gtt = read_text(card / "mem_info_vram_total"), read_text(card / "mem_info_gtt_total")
        if vram and gtt and vram.isdigit() and gtt.isdigit():
            info.setdefault("amdgpu", {})[card.parent.name] = {
                "vram_gib": round(int(vram) / GIB, 2), "gtt_gib": round(int(gtt) / GIB, 2)}
    pages = read_text("/sys/module/ttm/parameters/pages_limit")
    if pages and pages.isdigit() and "amdgpu" in info:
        info["ttm_pages_limit"] = int(pages)
        info["ttm_limit_gib"] = round(int(pages) * TTM_PAGE_BYTES / GIB, 2)
    if sys.platform == "darwin":
        disp = run(["system_profiler", "SPDisplaysDataType"], timeout=60)
        info["mac_gpu"] = first_line(disp, "Chipset Model")
        info["mac_gpu_cores"] = first_line(disp, "Total Number of Cores")
        wired = run(["sysctl", "-n", "iogpu.wired_limit_mb"])
        info["iogpu_wired_limit_mb"] = int(wired) if wired and wired.isdigit() else wired
        try:
            import mlx.core as mx  # noqa: PLC0415 - optional, Track M only
            dev = mx.device_info()
            for key in ("max_recommended_working_set_size", "memory_size"):
                if key in dev:
                    info["mlx_" + key] = dev[key]
        except (ImportError, AttributeError, RuntimeError):
            pass
    return info


def tool_versions():
    versions = {}
    for name, cmd, pick in (
        ("uv", ["uv", "--version"], ""),
        ("git", ["git", "--version"], ""),
        ("docker", ["docker", "--version"], ""),
        ("docker_compose", ["docker", "compose", "version", "--short"], ""),
        ("podman", ["podman", "--version"], ""),
        ("cmake", ["cmake", "--version"], ""),
    ):
        out = run(cmd)
        if out:
            versions[name] = first_line(out, pick)
    runtimes = run(["docker", "info", "--format", "{{range $name, $rt := .Runtimes}}{{$name}} {{end}}"])
    if runtimes:
        versions["docker_runtimes"] = runtimes.split()
    hf = shutil.which("hf")
    if hf:
        out = run([hf, "version", "--format", "json"])
        try:
            versions["hf"] = json.loads(out)["version"] if out else None
        except (ValueError, KeyError):
            versions["hf"] = None
    for module in ("torch", "mlx.core", "transformers", "huggingface_hub"):
        try:
            mod = __import__(module, fromlist=["__version__"])
        except ImportError:
            continue
        versions[module] = str(getattr(mod, "__version__", "unknown"))
        if module == "torch":
            versions["torch_cuda"] = getattr(mod.version, "cuda", None)
            versions["torch_hip"] = getattr(mod.version, "hip", None)
    return versions


def storage():
    models = Path.home() / "models"
    hf_home = os.environ.get("HF_HOME")
    usage = shutil.disk_usage(models if models.exists() else Path.home())
    return {
        "hf_home": hf_home,
        "hf_home_is_library": hf_home == str(models / "hf"),
        "hf_token_stored": bool(hf_home) and (Path(hf_home) / "token").exists(),
        "library_readme": (models / "README.md").exists(),
        "free_disk_gib": round(usage.free / GIB, 1),
    }


def build_record(track, note):
    return {
        "lab": "part-05/prepare-your-machine",
        "date": date.today().isoformat(),
        "track": track,
        "note": note,
        "machine": machine(),
        "python": platform.python_version(),
        "python_executable": sys.executable,
        "accelerator": accelerator(),
        "tools": tool_versions(),
        "storage": storage(),
    }


def main():
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--labbook", default=None,
                        help="append one JSON line to this existing file (for example labbook.md)")
    parser.add_argument("--track", default="auto", choices=TRACKS,
                        help="platform track; 'auto' guesses from what is installed")
    parser.add_argument("--note", default="", help="free text kept with the record")
    parser.add_argument("--print", dest="show", action="store_true",
                        help="print the record and write nothing")
    args = parser.parse_args()

    if args.labbook and not args.show and not Path(args.labbook).is_file():
        sys.exit(f"record-baseline: {args.labbook} does not exist. Run this from ~/llm-course, "
                 "where Part 1 created labbook.md, or pass its full path.")

    track = guess_track() if args.track == "auto" else args.track
    record = build_record(track, args.note)
    print(json.dumps(record, indent=2, sort_keys=True))

    if args.show or not args.labbook:
        if not args.labbook:
            print("\nNothing written: pass --labbook labbook.md to record this.")
        return

    with Path(args.labbook).open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(record, sort_keys=True) + "\n")
    print(f"\nrecorded in {args.labbook}")


if __name__ == "__main__":
    main()

Download record-baseline.py258 lines

Download it into ~/llm-course with the link under the listing, then run it in the same shell (the course environment is active from the block above):

RunnableAll tracks

append the baseline to the lab notebook

python record-baseline.py --labbook labbook.md --note "Part 5: prepared"

Output — what you should see

{
  "accelerator": { ... },
  "date": "2026-09-13",
  "lab": "part-05/prepare-your-machine",
  "machine": { "arch": "...", "kernel": "...", "os": "...", "system_memory_gib": ..., ... },
  "note": "Part 5: prepared",
  "python": "3.12.x",
  "python_executable": "/home/you/llm-course/.venv/bin/python",
  "storage": { "free_disk_gib": ..., "hf_home": "/home/you/models/hf", "hf_home_is_library": true, ... },
  "tools": { "docker": "...", "docker_compose": "...", "hf": "1.30.x", "torch": "...", "uv": "uv 0.12.11 (...)", ... },
  "track": "nvidia"
}

recorded in labbook.md

What lands under accelerator depends on the track:

Track	Keys under accelerator	Where each comes from
S	nvidia_smi, nvcc	nvidia-smi --query-gpu=name,driver_version,memory.total; the release line of nvcc --version
X	amdgpu (vram_gib, gtt_gib per card), ttm_pages_limit, ttm_limit_gib, vulkan (devices, drivers), rocminfo_gfx, amd_smi_version	sysfs, vulkaninfo --summary, rocminfo, amd-smi version
M	mac_gpu, mac_gpu_cores, iogpu_wired_limit_mb, mlx_memory_size, mlx_max_recommended_working_set_size	system_profiler, sysctl, mlx.core.device_info()
N	nvidia_smi, nvcc	as Track S; under WSL2 the script also looks in /usr/lib/wsl/lib

If track says unknown, rerun with --track and your track’s name. Finally, update the Machine section of labbook.md by hand with what no script can know, keeping Part 1’s fields and adding the GPU-memory line this lab created. One example, for Track X:

Fragment — not complete on its own

labbook.md, Machine section after Part 5, one example

## Machine

- Track: X
- Machine: <vendor and model>, Ryzen AI Max+ 395, Radeon 8060S, 128 GB, 2 TB NVMe
- Operating system and version: Ubuntu 24.04.x, kernel 6.17.0-10xx-oem
- Driver, CUDA, ROCm or macOS version: inbox amdgpu; Mesa RADV <version>; ROCm 7.2.1
- GPU memory limit: amd-ttm --set 113 = 29,622,272 pages; GTT 113.00 GiB; VRAM carve-out 0.50 GiB
- Containers: Docker <version>, Compose <version>
- Date prepared: 2026-09-13; baseline JSON appended the same day

On Track M the limit line is the iogpu.wired_limit_mb value and MLX’s cap; on Track S it is MemTotal; on Track N it is memory.total from nvidia-smi.

Preserve a recoverable machine baseline

Follow the platform tasks in order and retain the pre-change snapshot separately from the final baseline. If you already have a working accelerator stack, first identify it; do not replace it merely because another track’s command appears nearby.

At each layer, record the observable result: operating-system identity, driver/device enumeration, runtime availability, a real compute operation and model-storage permissions. Stop at the first failed layer. A successful container launch cannot repair a host driver that does not expose the device, and an accessible model directory does not establish sufficient free space for downloads and intermediate exports.

Before calling the machine ready, open a fresh terminal and repeat environment activation and the device check. This catches configuration that existed only in the original shell. Confirm that the model-storage location is the one later scripts will use and that it remains available after a reboot if it is a mount. Keep any manual adjustment beside the baseline so the capstone’s rebuild procedure does not depend on an undocumented BIOS, driver or shell setting.

Validation

You are done when every row passes, each checked by a command rather than from memory. Run them in a new terminal with the course environment active.

Check	Command	Pass
Nothing the course needs is missing	bash prepare-<track>.sh; echo $?	last lines summary: 0 MISSING, ... and 0
The accelerator answers	Track S, N: nvidia-smi -L; X: vulkaninfo --summary; M: system_profiler SPDisplaysDataType	your GPU is named (on X, a Radeon or RADV device of type INTEGRATED_GPU)
Track X: ROCm sees the GPU	rocminfo | grep -E "Name:\s+gfx"	gfx1151
Track X: the limit took	cat /sys/module/ttm/parameters/pages_limit	the pages amd-ttm --set printed
Track M: the limit took	sysctl -n iogpu.wired_limit_mb	the value you set, or 0 if the decision rule said leave it
Tracks S, N: the compiler is there	nvcc --version	release 13.x
Containers reach the GPU	the container test from task 6 for your track (not Track M)	the GPU table (S, N on Linux), the GPU Device 0: line naming your card (N on WSL2), or Hello from Docker! (X)
uv is the pinned one	uv --version	uv 0.12.11
hf reads the library	hf env | grep HF_HUB_CACHE	/home/you/models/hf/hub (/Users/you/... on a Mac)
The login, if you have one	hf auth whoami	✓ Logged in then user: <your username>
The notebook has the baseline	grep -c '"lab": "part-05/prepare-your-machine"' labbook.md	1 or more

Expected outcome

A machine every later part can assume, a check report ending summary: 0 MISSING beside the one you started from, and a dated baseline in the notebook.

Troubleshooting

Symptom	Cause	Fix
prepare-<track>: unknown argument	the check scripts take no options except --help	run bash prepare-<track>.sh with no arguments
Track S: nvidia-smi shows Memory-Usage: Not Supported	documented behaviour on the GB10’s integrated GPU	nothing; the per-process listing still shows memory in use
Track S or N: docker run --runtime=nvidia ... fails with unknown or invalid runtime name: nvidia, or docker run --gpus ... fails with could not select device driver "" with capabilities: [[gpu]]	the NVIDIA runtime is not registered with Docker, or the NVIDIA Container Toolkit is missing	the DGX Spark guide’s checks: nvidia-ctk --version, cat /etc/docker/daemon.json, then sudo nvidia-ctk runtime configure --runtime=docker and sudo systemctl restart docker; on Track N, if nvidia-ctk is not found, run task 6’s toolkit block first
docker info fails with a permission error about the Docker socket, or the check script reports docker access	not in the docker group in this session	id -nG must list docker; if not, sudo usermod -aG docker $USER, then log out and back in, or restart: a new terminal window is not enough
Track S: E: Unable to locate package docker-compose-plugin	DGX OS’s apt sources lack Docker’s repository	add Docker’s repository with the first block of task 6’s Track X tab (everything before sudo apt install docker-ce ...), then rerun the Compose install
Track N: the check script says Linux GPU driver ... installed inside WSL	an NVIDIA driver package was installed in Ubuntu under WSL2, which the CUDA on WSL guide forbids	remove the packages it names with sudo apt remove, run wsl --shutdown in PowerShell, reopen Ubuntu
Track N: nvidia-smi not found inside WSL2 but the card works in Windows	the WSL kernel or Windows driver is old, or the tool is not on PATH	wsl --update in PowerShell; the guide gives /usr/lib/wsl/lib/nvidia-smi as its location
Track N: driver still below 580 after ubuntu-drivers install	the recommended package is an older branch	sudo ubuntu-drivers list, then install a 580-or-later entry by name and restart
Tracks S, N: nvcc: command not found after task 3	/usr/local/cuda/bin is not on PATH in this shell	rerun task 3’s PATH block, or open a new terminal
Track X: uname -r does not end in -oem after the restart	the bootloader started the previous kernel, or the install did not finish	rerun sudo apt install linux-oem-24.04c and read its output for errors; restart and pick the OEM kernel in the boot menu if one appears
Track X: the check script reports amdgpu-dkms installed	a DKMS driver from a previous ROCm install; AMD’s Ryzen page requires the inbox driver	AMD’s page: sudo apt autoremove amdgpu-dkms dkms, then dkms status must print nothing
Track X: rocminfo lists no gfx agent, or reports a permission problem	render and video membership not yet in effect	groups; if either is missing, rerun the usermod line and restart
Track X: vulkaninfo --summary lists only llvmpipe	Mesa’s AMD driver is missing, or this session cannot open the GPU’s render node	sudo apt install mesa-vulkan-drivers; check groups for render, then log in again
Track X: amd-ttm: command not found	pipx ensurepath changed the profile, not the current shell	open a new terminal
Track X: pages_limit changed but mem_info_gtt_total did not	an older kernel whose amdgpu does not size GTT from the TTM limit	boot the OEM kernel from task 2; the 6.14 and 6.17 sources both size GTT from it
Track X: /dev/kfd absent	not established for this page; ROCm opens this device, and AMD’s udev rules name it	boot the OEM kernel AMD documents (task 2) and rerun the check; without it ROCm and Part 7’s ROCm container cannot reach the GPU
Track M: uname -m prints x86_64	the terminal runs under Rosetta	quit it and open a terminal application that is not set to run under Rosetta
Track M: sysctl reports an unknown name for iogpu.wired_limit_mb	macOS too old for the setting	update macOS (task 2); MLX documents the function as useful from 15.0
Track M: desktop freezes after raising the wired limit	the limit leaves macOS too little memory	restart, which the Apple silicon lesson says clears the setting; set a lower value
hf auth whoami says Error: Not logged in after task 7, but worked before	the token is still under ~/.cache/huggingface	rerun bash setup-model-storage.sh, which copies it, or hf auth login again
hf: command not found	the course environment is not active	source ~/llm-course/.venv/bin/activate
The task 7 download lands under ~/.cache/huggingface	this shell predates the profile change	open a new terminal; echo "$HF_HOME" must print the library path
record-baseline: labbook.md does not exist	run from a directory other than ~/llm-course	cd ~/llm-course, or pass the notebook’s full path

Cleanup

Everything this lab installed is used from here on: keep the drivers, toolkits, Docker, the environment, ~/models with ~/models/hf, both check reports and labbook.md. Three things are safe to tidy.

• The installer packages downloaded into ~/llm-course: cuda-keyring_1.1-1_all.deb (Track N) and amdgpu-install_7.2.1.70201-1_all.deb (Track X). They are already installed; delete the files if you like.
• The test file from task 7: hf cache rm model/Qwen/Qwen3-0.6B shows what it will delete and asks before deleting it.
• Test images: docker image rm hello-world (Track X), docker image rm ubuntu (Track N). Keep nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04 on Track S if you have the disk; later parts use NVIDIA’s images and re-downloading costs 3.9 GB.

⚠Warning

Two removals are destructive and only for someone who is sure. The old Hugging Face home: once hf auth whoami works and hf env shows the library path in a new terminal, rm -rf ~/.cache/huggingface deletes the original token and anything left in the old cache, and cannot be undone. Undoing the memory limits: amd-ttm --clear removes /etc/modprobe.d/ttm.conf and offers a restart back to the kernel default on Track X, but does not rebuild the initramfs: if amd-ttm --set printed The initramfs image needs to be regenerated, answer n to the restart, run sudo update-initramfs -u, then sudo reboot, and confirm with cat /sys/module/ttm/parameters/pages_limit. sudo sysctl iogpu.wired_limit_mb=0 returns Track M to the system default. Both make large models in later parts fail to load until the limit is raised again.

What you learned

• A visible GPU and a usable GPU are different observations. You saw the accelerator named by nvidia-smi, vulkaninfo, rocminfo or system_profiler, and then read the separate ceiling on what it may hold: VRAM on N, MemTotal on S, the TTM page limit on X, the wired limit on M.
• The memory ceiling is arithmetic you can check. On Track X, pages_limit × 4096 bytes matched mem_info_gtt_total and the kernel’s M of GTT memory ready line; on Track M, the sysctl read back the megabytes you set. The limit went into the Machine section.
• Versions are part of the machine. The kernel string, driver, toolkit release line, ROCm or macOS build, and the uv, Docker and hf versions are in the JSON baseline, and ROCm 7.2.1 was chosen because it matches the wheel, not because it is newest.
• Containers were proved, not assumed. A GPU table printed from inside a container on S and N (the GPU Device 0: line under WSL2), Hello from Docker! on X, nvidia among Docker’s runtimes on Track N’s Linux path, and docker in id -nG after a fresh login.
• One library, one Hugging Face home. hf env showed HF_HUB_CACHE and HF_TOKEN_PATH under ~/models/hf, the login survived the move, and a download landed in the library Part 4 created.
• The machine’s state is on file, twice. part05-after.txt ends summary: 0 MISSING, diff against part05-before.txt shows each MISSING line turning ok, and grep -c '"lab": "part-05/prepare-your-machine"' labbook.md counts the JSON line with its date, machine, accelerator, tools and storage sections.

Check your understanding

Question 1. A Track X machine reports MemTotal of 125.65 GiB. You run amd-ttm --set 100 and restart. What should /sys/module/ttm/parameters/pages_limit read, and what did it read before, at the kernel default?

• 100 before and after, because the parameter is in GB
• 26,214,400 after; about 16.47 million before
• 104,857,600 after, because the value is in KiB; about 65.9 million before
• 26,214,400 after; 32,938,393 before, because the default is all of memory

Show the answer and why

Answer: 26,214,400 after; about 16.47 million before

The limit counts 4,096-byte pages: 100 × 1024³ ÷ 4096 = 26,214,400. The kernel default is half of system memory, 125.65 ÷ 2 = 62.82 GiB, which is about 16.47 million pages (AMD shows 16,469,033). amdgpu then sizes GTT from that limit, so the log line after the restart reads 102400M of GTT memory ready.

Question 2. A reader who signed in during Part 2 runs these lines in order. Which one produces "Error: Not logged in"?

• export HF_HOME="$HOME/models/hf"
• mkdir -p ~/models/hf
• hf auth whoami, run after the export and before anything copies the token
• hf version

Show the answer and why

Answer: hf auth whoami, run after the export and before anything copies the token

huggingface_hub derives the token path from HF_HOME as $HF_HOME/token. The login was written to ~/.cache/huggingface/token, so once HF_HOME points elsewhere the stored token is simply not where hf looks. Copying token and stored_tokens across, as setup-model-storage.sh does, or logging in again fixes it; the export itself prints nothing.

Question 3. Part 1 installed torch 2.14.0+rocm7.2 on your Track X machine. AMD's newest install page is for ROCm 10.0.0. Which ROCm should this lab install?

• ROCm 10.0.0, because newer libraries are faster
• ROCm 7.2.1 from AMD's Ryzen page, because the ROCm on the machine must match the wheel's ROCm to the minor version
• Both, side by side, so either wheel works
• None: the Vulkan driver provides ROCm

Show the answer and why

Answer: ROCm 7.2.1 from AMD's Ryzen page, because the ROCm on the machine must match the wheel's ROCm to the minor version

The PyTorch wheel carries libraries built for ROCm 7.2, and Part 1 checks torch.version.hip against the installed stack. The 10.0.0 page also asks you to uninstall 7.2.4 or older first. Vulkan is a separate path used by llama.cpp; it does not provide ROCm.

Question 4. On a native Ubuntu Track N machine, docker info --format with the Runtimes template prints "io.containerd.runc.v2 runc". What happens in Part 7, and what is the fix?

• Nothing; --gpus works without a registered runtime on every Linux machine
• GPU containers fail because Docker has no nvidia runtime; install the NVIDIA Container Toolkit, run sudo nvidia-ctk runtime configure --runtime=docker and restart Docker
• Install the CUDA toolkit inside the container image
• Reinstall the NVIDIA driver with the cuda-drivers metapackage

Show the answer and why

Answer: GPU containers fail because Docker has no nvidia runtime; install the NVIDIA Container Toolkit, run sudo nvidia-ctk runtime configure --runtime=docker and restart Docker

The driver on the host is fine; what is missing is the runtime registration. Part 7's preflight runs docker run --runtime=nvidia --gpus all, and Docker refuses a runtime name it has not been told about ("unknown or invalid runtime name: nvidia"). nvidia-ctk writes the nvidia runtime into /etc/docker/daemon.json, and after a restart the same template lists "io.containerd.runc.v2 nvidia runc". Under WSL2 with Docker Desktop the list can differ, which is why the lab uses Docker's own --gpus test there.

Question 5. A 64 GB Mac: gpu-limit.py prints a course limit of 54 GiB, and MLX reports max_recommended_working_set_size as 60,000,000,000 bytes. What should you do with iogpu.wired_limit_mb?

• Set it to 55296, because the course limit is always applied
• Leave it at 0: 60,000,000,000 bytes is 55.9 GiB, already above the course limit, and a lower sysctl would shrink it
• Set it to 65536 so the GPU can use all memory
• Set it to 54, because the sysctl is in GB

Show the answer and why

Answer: Leave it at 0: 60,000,000,000 bytes is 55.9 GiB, already above the course limit, and a lower sysctl would shrink it

Divide by 1024³: 60,000,000,000 ÷ 1,073,741,824 = 55.88 GiB, which is at or above 54. The decision rule only ever raises the cap. 65536 would leave macOS nothing, and MLX documents that the wired limit must stay strictly below total memory; the sysctl takes megabytes, so 54 would be 54 MB.

Question 6. Inside WSL2, which of these is safe to install?

• nvidia-open from NVIDIA's repository
• The cuda-drivers metapackage
• cuda-toolkit-13 from the wsl-ubuntu repository
• The Linux driver from Ubuntu's ubuntu-drivers tool

Show the answer and why

Answer: cuda-toolkit-13 from the wsl-ubuntu repository

The Windows driver is mapped into WSL2, and the CUDA on WSL guide says not to install any Linux display driver inside it: nvidia-open, cuda-drivers and ubuntu-drivers all install one. The toolkit only compiles and links CUDA programs, and on 2026-09-13 the WSL-Ubuntu repository index listed no driver package such as nvidia-open or cuda-drivers.

Check my answersClear