# Model library

This directory is the single place model weights live on this machine. Every engine in the
Local LLM course is pointed at it, so a model is downloaded once and used by llama.cpp,
Ollama, LM Studio, vLLM, SGLang and mlx-lm alike.

Fill in the machine line below and keep this file with the library.

- **Machine:**
- **Platform track:** S / X / M / N
- **Memory available to models (GB), and whether that is unified memory or VRAM:**
- **Library created:**

## Layout

```
~/models/
  README.md                     this file
  <publisher>/                  the Hugging Face namespace, spelled as the Hub spells it
    <model>/                    the repository name, spelled as the Hub spells it
      <file>                    the weights
      <file>.sha256             the checksum verified at download time
      .cache/huggingface/       hf's own record of what it downloaded; leave it alone
  hf/                           reserved: Part 5 points the Hugging Face cache (HF_HOME) here
```

For example:

```
~/models/
  unsloth/
    Qwen3-8B-GGUF/
      Qwen3-8B-Q4_K_M.gguf
      Qwen3-8B-Q4_K_M.gguf.sha256
  ggml-org/
    gpt-oss-20b-GGUF/
      gpt-oss-20b-MXFP4.gguf
      gpt-oss-20b-MXFP4.gguf.sha256
```

Two rules keep the library usable a year from now. Keep the namespace, because
`Qwen3-8B-GGUF` under `unsloth/` and under `Qwen/` are different conversions, with different
files, checksums and cards. And keep the file name unchanged, because it names the
quantisation, which is the thing you will need to know when a benchmark number in the lab
notebook does not match a later run.

The `.sha256` file is one line in the format `sha256sum --check` reads, the hash, two spaces,
the file name, so any machine can re-check the file without the course's scripts.

## Adding a model

Use the lab's script, which looks up the checksum the Hub publishes, downloads with the `hf`
CLI, verifies the file and writes the `.sha256` beside it:

```sh
bash fetch-model.sh unsloth/Qwen3-8B-GGUF Qwen3-8B-Q4_K_M.gguf ~/llm-course/labbook.md
```

Re-running the same command is how you recover from an interrupted download: a file `hf`
has recorded as complete is not downloaded again, one it has not is, and the checksum is
checked on every run. A file that fails the check is moved to `<file>.corrupt` together
with `hf`'s record of it, so that the next run downloads it afresh.

## Verifying the library

To re-check everything, for example after a disk problem or an unclean shutdown, run the
lab's audit script, which reads every `.sha256` under the library, prints one line per file
and exits non-zero if any file has changed:

```sh
bash verify-library.sh ~/llm-course/labbook.md
```

The same check without the script, on Linux and in WSL2:

```sh
find ~/models -name '*.sha256' -execdir sha256sum --check {} \;
```

On macOS, `shasum -a 256 --check` does the same job. The `hf` CLI can also check one
repository's directory against the Hub directly:

```sh
hf cache verify unsloth/Qwen3-8B-GGUF --local-dir ~/models/unsloth/Qwen3-8B-GGUF
```

## Licences

Downloading weights does not change the licence they came under. Record the licence for
each model here as you add it, so that the answer is on the same machine as the files. The
course reference set, as recorded in the course model reference:

| Model | Licence | Gated | Notes |
| --- | --- | --- | --- |
| Qwen3 (all sizes) | Apache-2.0 | no | GGUF from unsloth, a third-party conversion; Qwen also publishes its own |
| gpt-oss-20b, gpt-oss-120b | Apache-2.0 | no | GGUF from ggml-org, the llama.cpp organisation |
| Llama 3.1 8B Instruct | Llama 3.1 Community License | yes (meta-llama) | GGUF is a community conversion (bartowski); accept the licence on the meta-llama page first |
| | | | |

Apache-2.0 and MIT models need no further thought. Anything else, a company-named licence,
a Llama community licence, Gemma terms, has conditions, and the course's licence lesson in
Part 3 covers what they are.

## Pointing tools at this directory

Later parts set these up properly. The short version:

- **llama.cpp** takes a path to the `.gguf` file, so nothing needs configuring.
- **Ollama** keeps its own store by default; Part 7 covers importing a GGUF from here instead
  of downloading it a second time.
- **LM Studio** expects `publisher/model/file` under its own models directory; Part 7 links
  it to this tree.
- **Transformers, vLLM and SGLang** read the Hugging Face cache rather than this tree; Part 5
  sets `HF_HOME` to `~/models/hf` so that cache lives on the same disk.

## What not to put here

Fine-tuned adapters, quantisations you made yourself and merged models belong in a separate
tree, because this one is meant to hold files whose checksums match a published original.
Part 13 onwards puts your own artefacts under `~/models-local/`.