Skip to content
Local LLM Course

Apple Clusters: MLX Distributed and exo over Thunderbolt 5

Part 19 taught clustering that works everywhere and is deliberately undemanding of the network: layers split across machines, one message per layer boundary, tolerant of ordinary Ethernet. This part is the opposite case. Apple silicon is the one home platform where the vendor’s own framework ships a distributed backend, the cable between two machines is fast enough to carry tensor parallelism, and the operating system gained remote direct memory access as a feature you switch on rather than a card you buy.

That combination is unusual enough to be worth a part of its own. It is also the part of Level 4 where the marketing is loudest, which is why it ends with a page that turns the loudest claim into a measurement.

What you will be able to do

Section titled “What you will be able to do”

By the end of this part you should be able to:

  • name MLX’s four distributed backends, say which transport each uses and what each one needs from the machines and from macOS, and choose between them for a given pair of Macs;
  • write a host file for mlx.launch, understand why the ring backend wants addresses and the RDMA backend wants device names, and generate both with mlx.distributed_config;
  • explain the difference between tensor parallelism and pipeline parallelism as mlx-lm implements them, say which one a given model supports, and predict what each does to the download, to the memory on each machine and to the traffic on the cable;
  • state precisely what macOS 26.2 added, what has to be done once from Recovery, and which of the requirements are documented rather than measured;
  • run exo across a set of Macs, read the placements it proposes, and say when its automatic partitioning is the right answer and when you want mlx-lm directly;
  • bridge two Macs over Thunderbolt 5, serve a model that neither could hold alone, and record prefill, decode and link use with the same discipline as every other measurement in this course;
  • state the claim “four Mac Studios replace a GPU server” as a set of workloads, measure the ones you can reach, compute cost per million tokens for both sides from list prices and your own throughput, and report which side won which case.

What you need first

Section titled “What you need first”

Part 18, all of it. The vocabulary of tensor and pipeline parallelism, the arithmetic that turns a split into bytes on a cable, and the network lab that gave your machines names, keys and measured links are all assumed here rather than re-taught. Part 8’s MLX lesson, because this part starts where that one’s distributed preview stopped. Part 5’s Apple silicon lesson, because every capacity question in this part is a unified-memory question.

Part 19 is not a prerequisite, but it is the fallback: readers with one Mac and one machine of some other kind reach a working cross-platform cluster through llama.cpp RPC rather than through MLX, and the lab points there explicitly.

The lab’s primary path needs two Macs with Thunderbolt 5 on macOS 26.2 or later and 32 GB each. That is a demanding requirement, and this part is honest about it: the single-machine path runs two ranks on one Mac, which teaches the mechanics and none of the capacity, and the reality check can be read and partly reproduced by anyone with one Mac and one other machine.

How to work through it

Section titled “How to work through it”

Read the two lessons in order. The first is the mechanism: what MLX actually does between two Macs, and what has to be true for each backend to work. The second is the tool that hides the mechanism, and it is worth reading even if you intend to use mlx.launch directly, because exo’s placement previews are a good way to see the choices a partitioner has to make.

Then the lab, which is long and mostly setup. Budget an evening rather than an hour if you are enabling remote direct memory access for the first time, because part of the procedure happens in Recovery and cannot be done over SSH.

Finish with the reality check. It is the page to send to anyone who has told you that a stack of Macs replaces a server, and the point of it is not that they are wrong. On some workloads they are right, on others they are not, and the difference is entirely predictable from the arithmetic in Part 18 once you say which workload you meant.

Part21 of 27Level4 — Cluster ArchitectPages4Estimated time2.9 hours

1 of 4 pages in this part need two or more machines for their primary path, marked below. Every one of them has a single-machine path, so the part can be completed with one computer.

0 / 4 lessons in this part completed