NVIDIA DGX Spark: GB10, DGX OS and the aarch64 Caveat

By the end of this lesson you will be able to describe what is inside a DGX Spark and which of its properties change how you install software; say what DGX OS is and how it is updated; explain why the machine’s processor architecture means that some downloads which work on every other Linux box will not work here, and what to fetch instead; and describe how two Sparks are joined, which is previewed here and built in Part 18.

This is the Track S lesson. If you are on another track, read it anyway: the Spark is the reference machine for several later parts, and knowing what it does differently is how you read the Track S branch of every lab that follows.

What is in the box

A DGX Spark is a single-board computer built around NVIDIA’s GB10 Grace Blackwell superchip. The hardware overview in NVIDIA’s user guide describes a 20-core Arm processor, ten Cortex-X925 cores and ten Cortex-A725 cores, alongside a Blackwell-architecture GPU with 6,144 CUDA cores, fifth generation Tensor Cores and fourth generation RT Cores, sharing 128 GB of LPDDR5x unified system memory on a 256-bit interface. The guide gives the SoC’s thermal design power as 140 W, and lists storage as 1 TB or 4 TB of self-encrypting NVMe M.2.

Connectivity is where the machine shows its intent. Alongside four USB Type-C ports, one HDMI 2.1a output and one 10 GbE RJ-45 connector, there are two QSFP network connectors driven by a ConnectX-7 adapter. A desktop machine does not normally have those. They are there so that Sparks can be cabled to each other, which is the subject of the last section.

The Founders Edition is not the only way to buy this board. The course hardware reference lists the OEM systems built on the same GB10 design: Acer Veriton GN100, ASUS Ascent GX10, Dell Pro Max with GB10, Gigabyte AI Top Atom, HP ZGX Nano, Lenovo ThinkStation PGX and MSI EdgeXpert. Chassis, storage options and support contracts differ; the compute, the memory and everything in this lesson do not. If you own one of those, you are on Track S.

Vendor specification, not measuredDGX Spark, as reported by NVIDIA

Figure	Reported value	Where NVIDIA states it
Unified memory	128 GB LPDDR5x, 256-bit, 4266 MHz	Hardware Overview
Memory bandwidth	273 GB/s	Hardware Overview and product page
Peak AI throughput	Up to 1 PFLOP FP4	Product page specifications
CUDA cores	6,144	Hardware Overview
CPU	20-core Arm: 10x Cortex-X925 + 10x Cortex-A725	Hardware Overview
ConnectX-7 ports	2x QSFP, up to 200 Gb/s each	ConnectX-7 Networking

NVIDIA DGX Spark (GB10), DGX OS · no engine; NVIDIA documentation retrieved 2026-09-09 · no model loaded, not applicable · 0 tokens of context · 2026-09-09

Specification figures reported by NVIDIA on the pages cited in this page's sources. None of them was measured by this course. The second lab in this part measures the bandwidth figure and the FP4 and BF16 throughput on your own machine.

▤Memory budget

128 GB of unified memory at that bandwidth is the whole argument for this machine. It fits models that no consumer graphics card can hold, including the 120B-class mixture-of-experts models in the model reference, and it reads them at a rate closer to a mid-range card than to a top one. Capacity is the reason to buy it; bandwidth is the reason a smaller model on a fast card still types faster.

DGX OS and the stack above it

The machine ships with DGX OS, which NVIDIA’s documentation describes as “a customized Linux distribution that provides a stable, tested, and supported operating system foundation for running AI, machine learning, and analytics applications on DGX systems”, based on Ubuntu. The course hardware reference records the current series as DGX OS 7.x on an Ubuntu 24.04 base.

The practical consequence is that you do not install a driver, a CUDA toolkit or a container runtime on a Spark. They arrive with the machine, matched to each other, and your job is to keep them current rather than to assemble them.

The Track S stack, top to bottom

what you install

1. Your code and the course scriptsPython in a uv environment, or a shell inside a container.yours
2. Engines and frameworksllama.cpp, vLLM, SGLang, TensorRT-LLM, PyTorch. Installed as arm64 wheels or pulled as arm64 container images.yours
3. CUDA toolkit and librariesDelivered with DGX OS and updated with it. Also available at newer versions inside NGC containers.
4. NVIDIA container toolkit and DockerPreinstalled and preconfigured, per the user guide.
5. NVIDIA driverPart of DGX OS. Not installed separately.
6. DGX OS 7.x, Ubuntu 24.04 base, aarch64The customised distribution NVIDIA supports on this hardware.
7. GB10 Grace Blackwell: 20 Arm cores, Blackwell GPU, 128 GB unified memoryOne package, one memory pool, one thermal budget.

Everything from DGX OS downwards is delivered and versioned by NVIDIA. When something fails to load a model, the layer to suspect is usually the one you installed yourself, which on this machine is only the top two.

The update guide gives a manual path and a preferred one. Its own advice is to “always prefer the DGX Dashboard for system updates”; the commands below are what the dashboard is doing, and they are what you need over SSH on a headless machine.

RunnableTrack S · DGX Spark

update DGX OS and firmware, as documented in the update guide

sudo apt update
sudo apt dist-upgrade
sudo fwupdmgr refresh
sudo fwupdmgr upgrade
sudo reboot

⚠Warning

sudo apt dist-upgrade can remove packages to resolve dependencies, and fwupdmgr upgrade writes firmware. Read what each proposes before confirming, and do not run the firmware step over a link you might lose. This is the one place in Level 2 where an interrupted command can leave a machine that does not boot.

Containers are the other half of the stack, and on this machine they are ready when you unbox it. The user guide states that “The NVIDIA Container Toolkit is preinstalled and configured on DGX Spark systems”, and gives this as the check that a container can reach the GPU:

RunnableTrack S · DGX Spark

confirm a container can see the GPU, from the user guide

docker run -it --gpus=all nvcr.io/nvidia/cuda:13.0.1-devel-ubuntu24.04 nvidia-smi

The aarch64 caveat

Here is the one architectural fact that changes every install step for the rest of the course.

The Spark’s CPU is Arm. Its Linux is aarch64, not x86_64. Almost everything in the Python and container ecosystem is published for x86_64 first and for aarch64 second, sometimes much later, and occasionally not at all.

What this means in practice:

• Python wheels. A pip install that silently succeeds on a desktop may find no matching wheel here and fall back to building from source, which for a package with compiled extensions can take an hour or fail on a missing toolchain. When a package offers an aarch64 wheel, you get it automatically; when it does not, you will know, because the install starts compiling.
• Container images. An image tag is not one image. A multi-architecture tag resolves to the right build for the machine that pulls it; a single-architecture tag does not, and pulling an x86-only image on a Spark either fails or, worse, runs under emulation at a fraction of the speed. NVIDIA’s own NGC images are published multi-arch, which is why the course reaches for them on this track.
• Prebuilt binaries. Release pages that offer “Linux” downloads frequently mean Linux on x86_64. Check for an aarch64 or arm64 asset before assuming a project ships one, and be ready to build from source, which on this machine is well supported and usually uneventful.

✕Common mistake

The symptom that catches people is not an error, it is silence followed by slowness. A container that started under emulation runs, produces correct output and is many times slower than it should be, and nothing on screen says so. If a workload on a Spark is inexplicably slow, check the image’s architecture before you check anything else.

Two checks turn all of that from a worry into a habit. The first tells you what the machine is; the second tells you what an image you are about to run was built for.

Fragment — not complete on its own

# What am I on?
uname -m                                    # aarch64 on a Spark

# What was this image built for? Run it before a long pull, not after a slow run.
docker image inspect --format '{{.Architecture}}' <image>

If the second command reports amd64 on this machine, you have found an x86-only image. Look for an arm64 tag, or use the NVIDIA-published equivalent, before you spend an hour wondering why a benchmark is disappointing.

✦Where to look first on this track

NVIDIA maintains a set of DGX Spark playbooks at https://build.nvidia.com/spark, covering inference with llama.cpp, vLLM, SGLang and TensorRT-LLM, fine-tuning with Unsloth, LLaMA-Factory, NeMo and PyTorch, and the multi-machine setups including connecting two Sparks, three Sparks, and several through a switch. They are written and tested against this exact hardware. When a course instruction and a current playbook disagree, the playbook is describing the machine and this course is describing the method; follow the playbook for the commands and tell us, because the page is then out of date.

FP4, and what Blackwell computes with

The GPU’s fifth-generation Tensor Cores compute directly with four-bit floating-point formats, which is what the peak-throughput row of the table above refers to. Part 1’s precision lesson introduced these formats; the hardware consequence is that on this track a four-bit model is not merely smaller in memory, it can also be multiplied natively rather than being unpacked to a wider format first.

Two cautions keep that from becoming a slogan. Native four-bit throughput helps prefill and training, which are compute-bound, far more than it helps decode, which is waiting on memory either way. And a format the hardware supports is not automatically a format your engine uses: whether a given build of llama.cpp, vLLM or TensorRT-LLM takes that path depends on the engine and the model file, which Parts 6 and 8 work through per engine. The right posture is the course’s usual one: the capability is documented, the benefit is a measurement.

Two Sparks, previewed

The two QSFP ports are the reason this machine appears in Level 4 at all. NVIDIA’s ConnectX-7 networking page states that “Each DGX Spark has two QSFP ports (sometimes called ‘ConnectX-7 ports’) on the back of the device” and that “Each port provides up to 200 Gigabits per second (Gb/s)”. It also states that “Each Ethernet interface has a corresponding RoCE interface (typically called a ‘RoCE device’) for InfiniBand communication”, which is what lets two machines move tensors between them without the operating system copying every byte through a network stack.

The documented limits are worth remembering before you buy cables. The page says the arrangement “supports up to three DGX Spark systems connected directly through cables, and up to four systems when using a switch”, and it names approved cable part numbers rather than leaving the choice open. NVIDIA’s product page frames the same capability from the model’s side: ConnectX networking “enables the connection of up to four NVIDIA DGX Spark systems to work with AI models of up to 700 billion parameters”.

⌂Single-machine path

Nothing in Level 2 needs a second Spark. One machine with 128 GB runs every lab in Parts 5 to 10 on its primary path. The pair matters from Part 18 onwards, and Part 18 states a single-machine path for every page that uses one.

Two documented surprises

NVIDIA’s known-issues page records two behaviours that look like faults and are not, both of them consequences of unified memory.

The first is that nvidia-smi “will display ‘Memory-Usage: Not Supported’ even though per-process GPU memory is listed”. On a discrete card, the tool reports how much of the card’s VRAM is in use; on a machine where the GPU’s memory is the system’s memory, that question has no separate answer, so the field is empty. Use the per-process listing, or the operating system’s own memory reporting, and do not conclude that the GPU is idle.

The second is that “the memory size reported by cudaMemGetInfo may be smaller than the actual allocatable memory”. Software that decides how much it can allocate by asking CUDA may therefore under-use the machine. When an engine on a Spark refuses a model you know fits, this is one of the first things to suspect, and Part 6’s challenge page walks through the diagnosis.

Separate architecture support from a runnable software stack

For Spark, a working deployment needs agreement across CPU architecture, operating system, accelerator driver, container image and model kernel support. A Linux container built only for x86 does not become an ARM image because both hosts use NVIDIA GPUs. A compatible CPU image can still contain kernels that do not cover the accelerator generation or requested dtype.

Build a stack record from the installed OS release, driver report, image digest, engine version and checkpoint identity. Start with a small known-supported checkpoint before allocating most of unified memory to a large one. This separates a loader or kernel incompatibility from a capacity failure.

When comparing with a desktop GPU, identify the resource you are buying: a larger shared memory pool, a particular software path or a network interface. None alone establishes faster interactive generation. Use the same application request and include sustained behaviour after warm-up. Your conclusion should state the workload Spark can host and the latency it delivers on your installation, rather than translating a low-precision compute headline directly into tokens per second.

Recap

A DGX Spark is a GB10 Grace Blackwell machine: an Arm CPU and a Blackwell GPU on one package, sharing 128 GB of unified LPDDR5x. Capacity is its reason to exist and bandwidth is its limit, both of them figures NVIDIA publishes and you will measure in the next lab. It runs DGX OS, an Ubuntu-based distribution that arrives with the driver, CUDA and the container toolkit already matched, so updates go through the dashboard or the documented apt and fwupdmgr sequence rather than through assembling a stack yourself.

The fact that reshapes every later install step is that the machine is aarch64: wheels, container images and release binaries all have to be the Arm build, and the failure mode when they are not is often slowness rather than an error. NVIDIA’s playbooks are the tested source for this hardware. The two ConnectX-7 QSFP ports, at the per-port rate quoted above and with a RoCE device per interface, are what make the two-machine cluster in Part 18 possible, within documented limits of three directly cabled systems or four through a switch.

Check your understanding

Question 1. A container image tagged "latest" pulls and runs on a Spark, but a workload that takes minutes on a colleague's desktop takes an hour. What should you check first?

• Whether the GPU driver is installed
• Whether the image is a multi-architecture tag, or an x86-only image running under emulation
• Whether the model file is corrupt
• Whether the machine is thermally throttled

Show the answer and why

Answer: Whether the image is a multi-architecture tag, or an x86-only image running under emulation

On aarch64 an x86-only image can run under emulation: the output is correct, nothing is printed to warn you, and the run is many times slower. NVIDIA publishes its NGC images multi-arch for this reason. Check the architecture of what you pulled before investigating anything else.

Question 2. On a DGX Spark, nvidia-smi reports "Memory-Usage: Not Supported". What does that mean?

• The driver is broken and should be reinstalled
• The GPU is not in use
• It is a documented behaviour on this unified-memory platform; the per-process listing still shows GPU memory use
• The CUDA toolkit version is too old

Show the answer and why

Answer: It is a documented behaviour on this unified-memory platform; the per-process listing still shows GPU memory use

The known-issues page documents it. On a machine where GPU memory and system memory are the same pool, there is no separate card memory to report. The per-process figures are still available.

Question 3. Which of these are true of the DGX Spark software stack as NVIDIA documents it? Select all that apply.

• The NVIDIA container toolkit is preinstalled and configured
• You must install the NVIDIA driver yourself before first use
• DGX OS is based on Ubuntu
• The documented manual update path uses apt and fwupdmgr, though the dashboard is preferred

Show the answer and why

Answer: The NVIDIA container toolkit is preinstalled and configured, DGX OS is based on Ubuntu, The documented manual update path uses apt and fwupdmgr, though the dashboard is preferred

The driver, CUDA and the container toolkit ship with DGX OS and are updated with it. The user guide states the container toolkit is preinstalled and configured, and the update guide gives the apt and fwupdmgr sequence while recommending the DGX Dashboard.

Question 4. How many DGX Spark systems does the ConnectX-7 networking page say can be connected directly with cables, and how many through a switch?

• Two directly, three through a switch
• Up to three directly, up to four through a switch
• Up to four directly, up to eight through a switch
• Any number, limited only by cable length

Show the answer and why

Answer: Up to three directly, up to four through a switch

The page states support for up to three systems connected directly through cables and up to four when using a switch. Each machine has two QSFP ports, each with a corresponding RoCE device for InfiniBand communication.

Question 5. True or false: because Blackwell Tensor Cores compute natively in FP4, running a four-bit model on a Spark will always decode faster than the same model in a format the hardware unpacks first.

• True
• False

Show the answer and why

Answer:

Native low-precision arithmetic mainly helps compute-bound work, which is prefill and training. Decode waits on memory, so what matters there is how many bytes are read per token. Whether an engine takes the native path is also a per-engine question, and either way it is a measurement rather than a promise.

Check my answersClear