Apple Silicon: Unified Memory, Metal and MLX

By the end of this lesson you will be able to read a Mac’s specification and say immediately which models it can hold and roughly how fast it will generate; explain what Metal and MLX are and how they relate; raise the one system limit that stops a large model loading on an otherwise capable machine; and say what this track is genuinely best at and where it stops.

This is the Track M lesson, and it applies to any Apple silicon Mac from the M4 generation onward, which is where this course draws its line.

Two machines, one idea

Apple silicon has had unified memory since the first M1, but the reason it appears in a course about running large models is a more recent development: the memory ceilings became large enough that a desktop purchase can hold a model that used to need a server, and the bandwidth scales with the chip rather than staying flat.

Apple’s own specification pages, read for this lesson on 2026-09-09, describe the current shape of the two desktops.

The Mac mini is the entry point. Its configurations pair a base chip with 16 GB or 24 GB of unified memory and Thunderbolt 4, and a Pro chip with 24 GB and Thunderbolt 5, with an Ethernet port Apple lists as “2.5Gb Ethernet port (configurable to 10Gb Ethernet)”. This is a machine for the 8 GB to 24 GB tiers of this course: the 8B-class models comfortably, the 14B class at a moderate quantisation, and every Level 1 and Level 2 lab on its primary or reduced path.

The Mac Studio is where the ceiling moves. Apple lists a Max chip configurable to 128 GB and an Ultra chip configurable to 256 GB or 512 GB, with four Thunderbolt 5 ports and 10Gb Ethernet. A 512 GB Mac Studio is, as far as this course’s hardware reference is aware, the largest pool of fast, GPU-addressable memory available as a single desktop purchase, and it is the reason the hardware reference’s top memory tier exists at all.

Vendor specification, not measuredApple silicon memory and bandwidth, as published

Chip	Unified memory as sold	Reported memory bandwidth GB/s	Where stated
M4	16-32 GB	120	Course hardware reference
M4 Pro	24-64 GB	273	Course hardware reference
M4 Max	36-128 GB	546	Course hardware reference
M5 Pro (Mac mini)	24 GB	307	Apple Mac mini specifications
M5 Max (Mac Studio, base GPU)	36-128 GB	460	Apple Mac Studio specifications
M5 Max (Mac Studio, 40-core GPU)	36-128 GB	614	Apple Mac Studio specifications
M3 Ultra	96-512 GB	819	Course hardware reference
M5 Ultra (Mac Studio)	96-512 GB	1,200	Apple Mac Studio specifications
M6 (Mac mini)	16-24 GB	153	Apple Mac mini specifications

Apple silicon Macs as configured by Apple, macOS · no engine; Apple specification pages and the course hardware reference, retrieved 2026-09-09 · no model loaded, not applicable · 0 tokens of context · 2026-09-09

Published figures, not measurements. Apple lists two bandwidth figures for the Mac Studio Max chip because the two GPU configurations differ; the Mac mini's base chip is listed at two figures for its two memory sizes, of which the larger is 170 GB/s. Measure your own machine in this part's second lab and record the result beside the row that applies to it.

Two patterns in that table are worth carrying away. Within this track, bandwidth rises with the chip tier and so does the memory ceiling, which is unusual: on Track N the biggest card and the fastest card can be the same speed and differ threefold in capacity. And the spread within the track is larger than the gap to any other track, which is why “a Mac” is never an answer to “will this run”.

Metal, MLX, and what runs on what

Two names cover almost everything on this track.

Metal is Apple’s graphics and compute API: the layer an engine talks to in order to run arithmetic on the GPU. It is the equivalent of CUDA’s role on the NVIDIA tracks and of Vulkan or HIP on Track X. llama.cpp’s Metal backend is enabled by default when it is built on macOS, which is why Track M’s build in Part 6 is the shortest of the four. PyTorch reaches the same hardware through its MPS backend, which is what Part 1’s lab used.

MLX is Apple’s array framework, built for this hardware from the start, and the native path for both inference and training on a Mac. Its documentation states the design directly: arrays “live in unified memory”, and “any device can perform any operation” on them “without needing to move them from one memory location to another”. You do not send an array to the GPU; you choose which device performs an operation, and the same bytes serve both. Parts 8 and 13 use MLX properly; this lesson only needs you to know that it exists and why it is fast here.

The Track M stack

what you install

1. Your code and the course scriptsPython in a uv environment.yours
2. Engines and frameworksllama.cpp with the Metal backend, mlx-lm, LM Studio, Ollama, PyTorch with the MPS backend.yours
3. MLX and Metal Performance ShadersThe array framework and the shader libraries that engines call on this hardware.
4. MetalThe GPU API. Part of macOS, versioned with it. There is no separate driver install and no CUDA-style toolkit.
5. macOS memory management, including the wired-memory limitDecides how much of the pool the GPU may hold resident. The one setting on this track that stops a model loading.the limit
6. Apple silicon: CPU, GPU and Neural Engine on one package, one memory poolBandwidth set by the chip tier, capacity set at purchase.

Nothing in the middle of this stack is installed by you: macOS ships Metal and the GPU driver together. What you install is the engine and the framework, which is why Track M setup is the shortest of the four and Track M troubleshooting almost always concerns memory rather than drivers.

The wired-memory limit

Here is the setting that Track M readers meet as a failure before they meet it as a concept.

macOS does not let the GPU hold arbitrary amounts of memory resident. There is a limit on wired memory, memory that cannot be paged out, and by default it is below the machine’s total. On a small model you never notice. On a model that occupies most of the machine, allocation fails or the system starts paging, and paging a language model is not a performance problem so much as an end to the experiment.

MLX documents both the limit and the way out. Its set_wired_limit function sets “the wired size limit” for MLX itself, in bytes, and its documentation notes that the function “is only useful on macOS 15.0 or higher”, that the default is 0, and that “the wired limit should remain strictly less than the total memory size”. Crucially, it also documents what to do when the limit you want exceeds the system’s own maximum:

Fragment — not complete on its own

# From MLX's set_wired_limit documentation: raise the system's wired-memory
# ceiling. <size_in_megabytes> is a number you choose; see the warning below.
sudo sysctl iogpu.wired_limit_mb=<size_in_megabytes>

The same documentation points at device_info() for the two numbers you need before choosing a value: "max_recommended_working_set_size", which it describes as the system’s wired limit cap, and "memory_size", the total. PyTorch exposes the equivalent through torch.mps, whose recommended_max_memory() “Returns recommended max Working set size for GPU memory in bytes”, with current_allocated_memory() and driver_allocated_memory() alongside it for seeing what is actually in use.

⚠Warning

sysctl iogpu.wired_limit_mb changes how much memory the whole system will pin for the GPU. Set it too close to the machine’s total and macOS has nothing left for everything else, which presents as a frozen desktop rather than as a clear error. Leave real headroom, change it while nothing important is running, and remember that it does not survive a reboot unless you make it persistent, which is a decision to take deliberately rather than by copying a command from a forum.

▤Memory budget

Why it bites at exactly the interesting sizes: the arithmetic says the model fits, so the reader concludes the machine can run it, and the machine disagrees for a reason that is nowhere in the arithmetic. Below, a 30B-class mixture-of-experts model at Q8 on a 64 GB Mac, with a 32,768-token key-value cache. It fits with room to spare, and it will still fail to load if the wired limit has not been raised.

Qwen3-30B-A3B at Q8_0 with 32k of context, on a 64 GB Mac

Weights, Q8_0

32.5 GB

Key-value cache, 32,768 tokens at FP16

3.2 GB

Free

28.3 GB

Total

64 GB

Sizes from the course model reference; the cache figure is the model's published bytes-per-token multiplied by the context length, as Part 4 taught. The free portion is what macOS keeps for everything else, and the wired-memory limit, not this arithmetic, is what decides whether the weights may be held resident.

Which macOS, and why the version is part of the specification

On the other three tracks the driver has a version of its own and the operating system is largely interchangeable. Here the operating system is the driver, so the macOS version is a specification number and belongs in the lab notebook beside the chip.

Three version boundaries matter for this course. MLX’s documentation says its wired-limit function “is only useful on macOS 15.0 or higher”, which sets the floor for anything that needs a large model resident. The course hardware reference names macOS 26, Tahoe, as the current series for Track M, and records that remote direct memory access over Thunderbolt 5, which the Level 4 Mac cluster uses, needs macOS 26.2 or later. And in general a Metal or MLX performance improvement arrives in a macOS release rather than in a driver package you can install separately, so “I updated macOS” is a legitimate entry in a benchmark log on this track and a legitimate explanation for a number moving.

The practical rule is the same one the rest of the course uses for versions: record what you were on, with the date, and when a figure changes, look at whether the operating system changed first.

Metal, MPS, MLX: three names, one GPU

The naming trips people up, so it is worth ten seconds. Metal is the API. MPS, Metal Performance Shaders, is Apple’s library of optimised kernels on top of it, and it is also the name PyTorch gives to its Apple silicon device, which is why the Part 1 lab printed accelerator: mps. MLX is a separate framework that also targets Metal but is not built on PyTorch at all. All three end at the same GPU cores and the same memory. When a project says it supports “Metal”, check which of the three it means, because a Metal backend in llama.cpp and an MLX implementation of the same model are different pieces of software with different performance and different bugs.

One thing none of them use: the Neural Engine. Apple silicon carries a separate neural accelerator, and it is genuinely useful for the on-device vision and speech work macOS does with it, but every engine this course teaches on Track M targets the GPU. If you see a Mac’s headline operations-per-second figure quoted in an argument about language models, check whether it is the GPU’s number or the Neural Engine’s, because they are not the same silicon and this course only uses one of them.

Thunderbolt 5, and the cluster preview

The Mac Studio’s four Thunderbolt 5 ports, which Apple’s specification page lists at a rate more than an order of magnitude above the machine’s 10Gb Ethernet, are the reason Track M appears in Level 4 alongside the Spark pair. Two or more Macs can be cabled directly to each other and run one model across them, and the course hardware reference records that remote direct memory access over Thunderbolt 5 requires macOS 26.2 or later.

That is all this lesson needs to say about it. Part 18 does the topology, the software and the single-machine alternative. What matters here is a purchasing point: if a second Mac is ever part of the plan, Thunderbolt 5 ports and a macOS version that supports RDMA over them are what make the plan work, and the Mac mini’s base configuration has Thunderbolt 4 rather than 5.

What this track is best at, and where it stops

Best at: the most model per desk. Nothing else in this course puts hundreds of gigabytes of GPU-addressable memory in a machine you can carry, at bandwidth that rises with the tier rather than staying flat. MLX is a genuinely good framework, mature for both inference and fine-tuning, and the Metal backend of llama.cpp needs no build flags to find the GPU. Set-up is the easiest of the four tracks; there is no driver to match to a kernel.

Where it stops: there is no CUDA, and a large amount of the machine-learning ecosystem assumes CUDA. Where a project publishes a CUDA-only kernel, a Track M reader waits for an MLX or Metal implementation or does without. Part 8 and Part 14 both mark the places where this bites.

Serving is a mixed picture and worth stating with a date. vLLM’s installation documentation, read on 2026-09-09, lists Apple silicon among its platforms, through a Metal-specific build, and separately lists Apple silicon among its CPU platforms. The course’s own hardware reference still records vLLM as not running on macOS, which is the older position; where the two disagree, the vendor’s current page is the one to check before you plan a lab. The course’s Track M serving path remains mlx-lm and llama.cpp, taught in Parts 7 and 8, because those are what the course has written and validated procedures for.

✱Hardware note

One more practical note that costs people a weekend: on a Mac, “unified memory” is bought once. There is no upgrade, no second slot, and no external accelerator. The memory decision at purchase is the whole capacity story for the life of the machine, which is why the hardware reference lists Track M by memory tier rather than by model name.

Understand what shared memory removes and what it retains

Unified memory reduces the need to copy some data between separate CPU and GPU memory pools. It does not eliminate memory bandwidth limits, operating-system pressure or the cost of materialising intermediate tensors. A model and the desktop applications still compete for a finite resource.

Read three observations together: process memory, overall memory pressure and swap activity. A process can appear to fit while the system compresses or swaps other pages, making a short benchmark misleading. Warm up the model and repeat a representative request while observing the whole machine. Record sustained latency, not just the first successful reply.

When comparing MLX with a Metal-backed engine, keep checkpoint lineage, precision, context and prompt formatting explicit. A converted MLX checkpoint and a GGUF of a different quantisation do not isolate engine efficiency. Start with a compatibility comparison if equal representations are unavailable, and label it accordingly. The useful question is whether a complete local application remains responsive within your memory budget while serving the workload you actually intend to use.

Recap

An Apple silicon Mac is unified memory taken furthest: one pool addressed by CPU and GPU with no copies, sized at purchase from tens to hundreds of gigabytes, at a bandwidth that rises with the chip tier. Metal is the GPU API and ships with macOS, so there is no driver to install; MLX is Apple’s array framework and the native path for inference and training, with llama.cpp’s Metal backend and PyTorch’s MPS backend as the alternatives.

The one setting that has to be understood is the wired-memory limit. macOS caps how much memory may be held resident for the GPU, MLX documents both the framework-level function and the iogpu.wired_limit_mb sysctl that raises the system ceiling, and both MLX and PyTorch expose the recommended maximum so you can choose a value rather than guess. Thunderbolt 5 with RDMA on a recent macOS is what makes the Level 4 Mac cluster possible. The track’s ceiling is the absence of CUDA, and the places where that costs you are marked where they occur.

Check your understanding

Question 1. A 64 GB Mac has a model whose weights and key-value cache total about 36 GB. It fails to allocate. What is the first thing to check on this track?

• That the model file downloaded correctly
• The wired-memory limit, which caps how much memory macOS will hold resident for the GPU and defaults to less than the machine total
• Whether Metal is installed
• The Thunderbolt firmware version

Show the answer and why

Answer: The wired-memory limit, which caps how much memory macOS will hold resident for the GPU and defaults to less than the machine total

The arithmetic said it fits and the machine disagreed, which on Track M points at the wired limit rather than at capacity. MLX documents the sysctl that raises the system ceiling and the device_info fields that tell you the current cap and the total.

Question 2. What does MLX's documentation say about where its arrays live?

• They live on the CPU until you copy them to the GPU
• They live in unified memory, and any device can operate on them without moving them between memory locations
• They live in a Metal-only heap that the CPU cannot read
• They are paged from disk on demand

Show the answer and why

Answer: They live in unified memory, and any device can operate on them without moving them between memory locations

That is the design that makes this hardware suit large models: no device-to-device copies, and one pool sized at purchase. You choose which device runs an operation, not where the data sits.

Question 3. Which of these are true of Apple silicon as a track for this course? Select all that apply.

• Memory is fixed at purchase and cannot be upgraded
• Bandwidth generally rises with the chip tier, alongside the memory ceiling
• CUDA is available through a compatibility layer
• There is no separate GPU driver to install and match to a kernel

Show the answer and why

Answer: Memory is fixed at purchase and cannot be upgraded, Bandwidth generally rises with the chip tier, alongside the memory ceiling, There is no separate GPU driver to install and match to a kernel

Memory and chip tier are one decision taken at purchase. Metal ships with macOS, which is why Track M setup is short. There is no CUDA on this hardware, and where a project ships CUDA-only kernels, Track M waits for a Metal or MLX implementation.

Question 4. Why does this lesson give a date when describing which platforms vLLM supports?

• Because dates are required in all frontmatter
• Because platform support changes between releases, and an undated support claim becomes folklore; the course records what a page said and when it was read
• Because vLLM changes its licence often
• Because Apple requires it

Show the answer and why

Answer: Because platform support changes between releases, and an undated support claim becomes folklore; the course records what a page said and when it was read

This lesson found the vendor page and the course's own hardware reference disagreeing about macOS support. Recording the page and the date is what lets a later reader see which one aged and check for themselves.

Check my answersClear