Managing a Model Library: Storage, Naming and Versions

By the end of this lesson you will have one copy of each model on your disk instead of three, every engine in this course reading from it, a record beside each file saying where it came from and what it hashes to, and a rule for deciding what to delete that is better than “whatever is biggest when the disk fills up”.

This is the least exciting page in this part. It is also the one that decides whether Part 9 is a pleasant afternoon or a day of re-downloading things you already have.

The problem, stated plainly

By the end of Part 9 you will have four programs that all want the same weights, and each of them has its own idea of where weights live:

• llama.cpp reads whatever path you give --model. It has no library at all, which makes it the easiest to satisfy.
• Ollama keeps a content-addressed store of blobs plus one manifest per tag, at ~/.ollama/models on macOS, /usr/share/ollama/.ollama/models on Linux and C:\Users\%username%\.ollama\models on Windows.
• LM Studio expects ~/.lmstudio/models/publisher/model/model-file.gguf — three levels, publisher then model then file.
• The Hugging Face cache, which transformers, mlx-lm, vLLM and the hf command all share, at ~/.cache/huggingface/hub by default.

Left alone, a household that experiments with all four ends up with the same 5 GB file four times, under four names, with no record of which conversion any of them is. The fix is one directory and three ways of pointing at it.

One download, four readers

1. 1Download once, verifiedhf download into ~/models/<publisher>/<repository>/, checksum checked against the Hub, .sha256 written beside the file. This is Part 4's fetch-model.sh.
2. 2llama.cpp reads the pathllama-server --model ~/models/Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q4_K_M.gguf. Nothing to configure.
3. 3LM Studio reads a symbolic linkA link at ~/.lmstudio/models/Qwen/Qwen3-8B-GGUF/ pointing into the library satisfies the publisher/model/file layout without a second copy.
4. 4Ollama reads a ModelfileFROM ./Qwen3-8B-Q4_K_M.gguf imports the file into its own store once, under a name you chose, with the template and parameters you chose.
5. 5Everything Python reads the cacheHF_HOME or HF_HUB_CACHE moves the shared cache onto the same disk, so the safetensors originals are somewhere you can find and prune.

Planning the disk before you fill it

Storage plans go wrong because people budget for the model they want and not for the four they will try. A workable rule is to plan for three models at your tier plus one tier above, which is roughly three times the size of your usual model, and to keep the small one permanently because it costs almost nothing and answers most of your “does this script work” questions.

Memory tier	A sensible resident set	Rough library size
8 GB	Qwen3-1.7B Q8_0, Qwen3-4B Q4_K_M	under 5 GB
12–16 GB	the above plus Qwen3-8B Q4_K_M	about 10 GB
24 GB	the above plus Qwen3-14B Q4_K_M	about 20 GB
32 GB	the above plus gpt-oss-20b MXFP4	about 35 GB
48–64 GB	the above plus Qwen3-30B-A3B Q4_K_M	about 55 GB
96–128 GB	the above plus one very large model	120 GB and up

Those are file sizes, from Part 4’s table of published sizes on the Hub. The number that actually fills a disk is larger, because the Hugging Face cache keeps the safetensors originals of anything you converted, and because a model kept in two formats — GGUF for llama.cpp and MLX for a Mac — is two files by necessity, not by accident.

✦Tip

Put the library on the largest disk, not the fastest one. Weights are read once per load, sequentially, and then live in memory; the load time difference between a good SSD and a very good one is seconds per session. Free space is the constraint that actually stops you working.

The Hugging Face cache, which is not a directory of models

Anyone who has looked inside ~/.cache/huggingface/hub and given up deserves an explanation. The documentation gives one: each repository becomes a folder named models--<org>--<name>, and inside it are four directories with different jobs.

• blobs holds “the actual files that we have downloaded. The name of each file is their hash.”
• snapshots holds one folder per revision, containing symbolic links with the real file names, pointing at the blobs. This is the layout that lets two revisions of a repository share an unchanged file instead of storing it twice.
• refs records which commit a branch name currently points at.
• trees caches the file list of a commit, so re-downloading something already cached costs a single network call rather than one per file.

Two practical consequences. Deleting things by hand from inside that tree breaks the bookkeeping, which is why the CLI has its own delete commands. And on Windows, where symbolic links need Developer Mode or an administrator, the documentation says the library “does not use the blobs/ directory but directly stores the files in the snapshots/ directory instead”, so the same file downloaded at two revisions is stored twice. If you are on Track N under Windows and the cache is larger than the sum of your models, that is why.

The cache also writes a CACHEDIR.TAG file, which “tells backup tools (e.g. Borg, restic, rsync) that the directory contains re-downloadable cache data and can safely be excluded from backups”. Honour it. Backing up sixty gigabytes of files that a single command can fetch again is a waste of the backup you actually need, which is the one in this part’s lab.

RunnableAll tracks

move the shared cache onto the library disk

export HF_HOME=/data/hf
hf download Qwen/Qwen3-4B-GGUF Qwen3-4B-Q4_K_M.gguf --local-dir ~/models/Qwen/Qwen3-4B-GGUF

HF_HOME moves the whole Hugging Face home; HF_HUB_CACHE moves only the hub cache. Set it in your shell profile, not per command, or you will end up with two caches and no idea which is which.

Making the other tools read your library

LM Studio: a link in the right shape

LM Studio’s requirement is a path shape, not a file format, and a symbolic link satisfies a path shape:

RunnableTrack M · Apple silicon

point LM Studio at the shared library

mkdir -p ~/.lmstudio/models/Qwen
ln -s ~/models/Qwen/Qwen3-8B-GGUF ~/.lmstudio/models/Qwen/Qwen3-8B-GGUF
lms ls

The same works on Linux. On Windows, a link needs Developer Mode or an elevated shell, for the same reason the Hugging Face cache degrades there; lms import copying the file is the supported alternative, at the cost of a second copy.

Ollama: a Modelfile, or a different store

Ollama will not read a loose GGUF file from an arbitrary path at run time, but the Modelfile reference documents FROM ./ollama-model.gguf as a way to build a model from one:

RunnableAll tracks

import a library file as an Ollama model

printf 'FROM %s/models/Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q4_K_M.gguf\nPARAMETER num_ctx 8192\n' "$HOME" > Modelfile
ollama create qwen3-8b-local -f Modelfile
ollama list

That imports the weights into Ollama’s own store, so it is a copy — the honest trade is that you get Ollama’s scheduler and one extra copy of one model, rather than a copy of everything. If you would rather have Ollama’s whole store on the library disk, OLLAMA_MODELS moves it, and on Linux the FAQ is explicit that the service user needs access: sudo chown -R ollama:ollama <directory>.

✕Common mistake

Symbolically linking Ollama’s blob directory at some other tree is the tempting shortcut and it is a good way to lose models. The store is content-addressed with manifests that reference blobs by hash; half-moving it leaves manifests pointing at blobs that are not there, and the symptom is a model that lists fine and refuses to run. Move the whole store with OLLAMA_MODELS, or import individual files with a Modelfile. Do not mix the two.

Naming and recording, so that six months later you know what you have

Part 4 set the rule: keep the publisher’s namespace in the path, so ~/models/Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q4_K_M.gguf rather than ~/models/qwen3-8b.gguf. Three things belong beside the file, and none of them is expensive:

1. The checksum. A .sha256 file next to the weights, written at download time, is what turns “the model started behaving oddly” into a question you can answer in a minute.
2. The provenance. Which repository, which revision, which day. A community quantisation is a legitimate artefact and it is not the publisher’s file; when your benchmark disagrees with somebody else’s, this is the first difference to check.
3. The reason. One line in the library README saying what this model is for on this machine. Future you will not remember why there are two 8B models.

The hf CLI can check the first of those against the Hub for a whole directory at once. hf cache verify Qwen/Qwen3-8B-GGUF --local-dir ~/models/Qwen/Qwen3-8B-GGUF validates every local file in that directory against the checksums the Hub publishes and exits non-zero on a mismatch:

Output — what you should see

Verified 13 file(s) for 'Qwen/Qwen3-8B-GGUF' (model) in ...
  All checksums match.

Pruning, with a rule instead of a panic

Deleting models is the part people do badly, usually at midnight when a download has failed for lack of space. Decide the rule in advance.

A rule that works: keep anything you have benchmarked, anything smaller than 3 GB, and anything you used this month; delete the rest, largest first. Benchmarked models are worth keeping because a comparison across time needs the same artefact at both ends. Small models are worth keeping because they cost nothing and save minutes every day. Everything else is one download away.

Start by seeing what you have, in each of the three stores:

RunnableAll tracks

what is on this disk, and how large

du -sh ~/models/*/* | sort -h | tail -20
ollama list
hf cache ls

hf cache ls aggregates by repository and shows size, last access and last modification, and it accepts filters such as --filter "accessed>30d", which is very close to the rule above written as a command.

⚠Warning

The next three commands delete weights permanently, and the only way back is to download them again. ollama rm removes a model from Ollama’s store; hf cache rm removes a repository or a revision from the shared cache; hf cache prune removes revisions no longer referenced by any branch or tag, plus leftover .incomplete files from interrupted downloads. Read the name in the command before you press return, and use --dry-run on the cache commands the first time.

Both cache commands accept --dry-run, which prints what would go and removes nothing, and --yes for when you are scripting it. Preview first, always; then delete the one you meant:

RunnableAll tracks

check the name, then remove it

ollama list
ollama rm qwen3-8b-local

hf cache prune is the safe one to run regularly: an unreferenced revision is by definition one that no branch or tag points at any more, and the .incomplete files it removes are partial downloads that nothing will ever use. hf cache rm and ollama rm are the ones to read twice.

▤Memory budget

A last piece of arithmetic, because it decides more pruning decisions than file size does. The weights are only part of what a model costs you at run time: Part 4’s key-value cache formula adds roughly 1.2 GB for Qwen3-8B at 8,192 tokens and roughly 4.8 GB at 32,768. A model that fits on disk five times over may not fit in memory once at the context length you want. Prune for the disk, but choose the resident set for the memory.

Make promotion and rollback operations explicit

Use separate concepts for a downloaded candidate, an evaluated artefact and an active service alias. A download becomes a candidate after integrity and provenance checks. It becomes eligible for use after the task evaluation. Updating the alias is the promotion step, and the previous mapping is the rollback record.

For every promoted model, retain the source repository and revision, exact file hashes, licence, conversion recipe, tokeniser/template identity and evaluation result. Include context and memory limits in the alias’s deployment configuration. An alias called “chat” should remain stable for clients even when the underlying checkpoint changes, but its history must remain inspectable.

Before removing an old model, check whether adapters, quantised derivatives or saved experiments depend on it. A small adapter can require a large exact base that you are about to delete. Rebuildable files may be candidates for removal; unique training data and evaluation records usually are not. Test rollback by restoring the previous alias mapping and repeating a representative request, rather than assuming that keeping a filename is sufficient.

Recap

• Four tools, four ideas of where models live: a bare path for llama.cpp, a content-addressed store for Ollama, a publisher/model/file.gguf tree for LM Studio, and the shared Hugging Face cache for everything Python.
• One library at ~/models, with the publisher namespace preserved, feeds all of them: a path for llama.cpp, a symbolic link for LM Studio, a Modelfile import for Ollama.
• The Hugging Face cache is blobs plus symbolic links, not a folder of models. Delete from it with hf cache rm and hf cache prune, never by hand, and never back it up: it carries a CACHEDIR.TAG asking you not to.
• On Windows the cache stores duplicates instead of links unless Developer Mode is on, which explains a great deal of unexplained disk usage on Track N.
• Record the checksum, the provenance and the reason beside every file. hf cache verify re-checks a whole directory against the Hub.
• Prune by a rule decided in advance: keep what you benchmarked, keep what is tiny, keep what you used this month, delete the rest largest first.

Check your understanding

Question 1. You want LM Studio and llama.cpp to use the same GGUF file without storing it twice. What works?

• Copy the file into ~/.lmstudio/models/
• Create a symbolic link inside ~/.lmstudio/models/ that reproduces the publisher/model layout and points at the library directory
• Set an environment variable in LM Studio pointing at ~/models
• It is not possible; each tool needs its own copy

Show the answer and why

Answer: Create a symbolic link inside ~/.lmstudio/models/ that reproduces the publisher/model layout and points at the library directory

LM Studio's requirement is a path shape — publisher, then model, then the .gguf file — and a link satisfies a path shape at no cost in disk. On Windows a link needs Developer Mode or an elevated shell, which is when lms import and a second copy become the pragmatic answer.

Question 2. Why does the course say never to delete files by hand from inside ~/.cache/huggingface/hub?

• The files are encrypted
• The cache is blobs plus symbolic links plus refs and tree metadata, and removing pieces of it by hand leaves the bookkeeping inconsistent; hf cache rm and hf cache prune exist for this
• It requires root
• The files are needed by the operating system

Show the answer and why

Answer: The cache is blobs plus symbolic links plus refs and tree metadata, and removing pieces of it by hand leaves the bookkeeping inconsistent; hf cache rm and hf cache prune exist for this

The same reasoning applies to Ollama's store, which is content-addressed blobs plus manifests. Both tools give you a delete command precisely because the on-disk layout is not the layout it appears to be.

Question 3. Your Windows machine has 40 GB of Hugging Face cache but only about 22 GB of distinct model files. What is the documented explanation?

• The cache stores every model twice as a backup
• Without Developer Mode or administrator rights the library cannot use symbolic links, so it stores files directly in snapshots/ and the same file at two revisions is stored twice
• Windows compresses files, inflating the reported size
• The cache includes the operating system's page file

Show the answer and why

Answer: Without Developer Mode or administrator rights the library cannot use symbolic links, so it stores files directly in snapshots/ and the same file at two revisions is stored twice

The limitation is stated in the caching guide, along with the two fixes: activate Developer Mode, or run as administrator. It is also why the same download can look much larger on Track N under Windows than on the same machine under WSL2.

Question 4. Which of these belong beside a downloaded model file, according to this lesson? Select all that apply.

• The SHA-256 checksum recorded at download time
• The repository and revision it came from
• A line saying what this model is for on this machine
• A copy of the model in a second quantisation, for comparison

Show the answer and why

Answer: The SHA-256 checksum recorded at download time, The repository and revision it came from, A line saying what this model is for on this machine

The first three cost bytes and answer questions you will genuinely have. A second quantisation is a legitimate thing to keep, but it is a resident-set decision made against your memory budget, not a record-keeping one.

Check my answersClear