AMD-Native: ROCm Builds, Lemonade Server and the NPU Question

By the end of this lesson you will be able to say what AMD’s own documentation claims about the Radeon 8060S today and on which date, build llama.cpp against HIP instead of Vulkan, run models through Lemonade Server and choose its backend, and answer the question every owner of one of these machines eventually asks: what is the NPU for, and can it run my model?

This lesson is written for Track X. The other tracks should read the section on reading vendor documentation, because the technique generalises and the AMD documentation happens to be the clearest worked example of why it is needed.

Four layers, and which of them is the uncertain one

The AMD stack on a Ryzen AI Max+ 395

each layer depends on the one below

1. Ryzen AI Max+ 395 (Strix Halo)Zen 5 cores, a Radeon 8060S integrated GPU on the gfx1151 target, an XDNA 2 NPU, and one pool of memory shared between them.fixed at purchase
2. Kernel driver and firmwareamdgpu for the GPU; a separate NPU driver, which on this chip is a Windows component.distribution or vendor
3. Compute runtimeVulkan, which every distribution ships, or ROCm and HIP, which you install from AMD.the uncertain layer
4. Enginellama.cpp built for Vulkan or HIP; PyTorch on ROCm for training work; ONNX Runtime GenAI for the NPU paths.
5. Server and clientsllama-server, or Lemonade Server wrapping several engines behind one OpenAI-compatible port.

Vulkan and ROCm are alternatives at the same layer, not a stack. The course's default on this track is Vulkan because it comes from ordinary distribution packages; ROCm is the path with more upside and more version sensitivity.

The layer marked as uncertain is uncertain for a documented reason, and reading the documentation honestly is the first skill this lesson teaches.

What AMD’s documentation says about this chip, and when

On 2026-09-09 three AMD pages, all reachable from the same documentation site, described the same software stack differently. This is not a criticism to be resolved by picking a favourite; it is the actual state of the material a reader has to work from, so the course records it.

Page	Version it documents	What it says about gfx1151	Date on the page
ROCm compatibility matrix	ROCm 10.0.0	Lists “AMD Ryzen AI Max+ 395 (Radeon 8060S) (gfx1151)” among supported GPUs, with no support-tier qualifier	Matrix dated 2026-08-14
System requirements (Linux)	ROCm 7.14.0	gfx1151, Radeon 8060S and Ryzen AI Max+ do not appear	Last updated 2026-07-15
Install PyTorch for ROCm	Validated with ROCm 7.2.4	Does not enumerate architectures; points back at system requirements	Last updated 2026-07-15

Three conclusions follow, and they are worth more than any single fact on the table.

The chip is supported by name somewhere authoritative. The compatibility matrix is AMD’s own, it names the exact part and the exact target, and it attaches no preview or experimental qualifier. That is a real change from a year ago and it is why the HIP build below is a reasonable thing to attempt.

The install path is not equally confident. The page a reader would land on when trying to install does not mention the chip. Whichever of these is stale, the consequence for you is the same: the install may need a package set that page does not cover, and you should expect to read release notes.

Everything on this table has a date. If you are reading this months later, re-check all three pages before believing any of the rows. The course records what it read and when, and asks you to do the same in the notebook.

⌚Version note

The habit worth taking from this table: when a vendor’s own pages disagree, write down which page you used, which version it documented, and the date. That line in your notebook is what lets you explain, six weeks later, why a command that worked stopped working. Guessing which page was right is not a technique.

Building llama.cpp against HIP

Part 6 built the Vulkan path and made it the course default on this track. The HIP path is the alternative, and the llama.cpp build guide gives the recipe. The GPU target is the piece that has to be right: the Radeon 8060S is gfx1151.

RunnableTrack X · Ryzen AI Max+

configure and build the HIP backend into its own directory

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
    cmake -S ~/llama.cpp -B ~/llama.cpp/build-hip \
    -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DCMAKE_BUILD_TYPE=Release
cmake --build ~/llama.cpp/build-hip --config Release -j "$(nproc)"

Three notes from the build guide itself. GPU_TARGETS is optional and omitting it builds for every GPU the machine reports, which takes longer and is the sensible choice only if you do not know your target. If the build fails with a device-library error, the guide’s fix is to find the directory containing oclc_abi_version_400.bc under HIP_PATH and prepend HIP_DEVICE_LIB_PATH=<directory> to the command. And keeping the HIP build in its own directory, as above, is what makes the comparison in this part’s lab possible: two builds, two servers, one machine.

RunnableTrack X · Ryzen AI Max+

prove which backend you just built

~/llama.cpp/build-hip/bin/llama-server --version
rocm-smi --showmemuse

The backend name in the server’s load log is the evidence, exactly as in Part 6. A HIP build that silently fell back to the CPU behaves like a correct one, only slower.

▤Memory budget

Both builds draw from the same pool, and the GPU-visible share of that pool is capped below the machine total: about 96 GB on Windows, and on Linux by the GTT limit, which has to be raised deliberately. Part 5’s lesson on this machine covers how to check and change it. A model that fits the machine but not the cap fails to allocate, which looks like a driver bug and is not one.

PyTorch on ROCm, briefly

Inference is not the only reason to want ROCm. The training parts of this course need PyTorch, and AMD documents two ways to get it: a prebuilt container, rocm/pytorch:latest, or wheels from PyTorch’s own ROCm index. The wheel command on the page read on 2026-09-09 was

Fragment — not complete on its own

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm7.2

with the page stating that PyTorch is validated against ROCm 7.2.4 on Ubuntu 22.04 and 24.04 with Python 3.10 and 3.12, and that builds cover “a broad set of AMD architectures” with PYTORCH_ROCM_ARCH available to narrow them. Note the version in that index URL against the versions in the table above. Part 11 returns to this when it sets up the training toolchain, and states the track’s status there as well.

Lemonade Server

Lemonade is an open-source local AI server, Apache-2.0 licensed, that wraps several inference engines behind one OpenAI-compatible endpoint and picks a backend for the hardware it finds. Its own description is “the local AI server that gives you the same capabilities as cloud APIs, except 100% free and private”. On this track its appeal is specific: it is the one server in this part that can address the GPU and the NPU through the same interface.

The engines it lists include llamacpp, with system, Metal, CUDA, Vulkan and ROCm backends; flm and ryzenai-llm for the NPU; whispercpp for speech to text; kokoro for speech synthesis; sd-cpp for images; and an experimental vllm on ROCm. Installation is a distribution package on Linux, an MSI on Windows and a .pkg on macOS. AMD’s own getting-started playbook gives the Ubuntu commands:

RunnableTrack X · Ryzen AI Max+

install on Ubuntu, from AMD's playbook

sudo add-apt-repository ppa:lemonade-team/stable
sudo apt install lemonade-server

Driving it is a small vocabulary. lemonade pull <MODEL_NAME> downloads a registered model, lemonade run <MODEL_NAME> loads it and opens the web application, lemonade list shows what is available and lemonade status reports whether the server is reachable. The server listens on port 13305 by default, with --host and --port as global options and the environment variables LEMONADE_HOST and LEMONADE_PORT as the other way to set them. The playbook gives the base URL as http://localhost:13305/api/v1, and the repository’s own client example uses http://localhost:13305/v1 for OpenAI-compatible clients.

The flag that matters on this track selects the llama.cpp backend explicitly:

RunnableTrack X · Ryzen AI Max+

choose the backend rather than accepting the detected one

lemonade run Qwen3-8B-GGUF --llamacpp rocm
lemonade run Qwen3-8B-GGUF --llamacpp vulkan

The playbook also documents lemonade config set llamacpp.backend=rocm for a persistent default and the environment variable LEMONADE_LLAMACPP, which accepts vulkan, rocm or cpu. That is the cheapest Vulkan-against-ROCm comparison available on this machine, and this part’s lab makes it a measurement.

!Important

Lemonade’s OpenAI compatibility is documented per endpoint rather than claimed as a whole. Its API page marks /v1/chat/completions and /v1/responses as partially available and /v1/completions, /v1/embeddings, /v1/models and the image and speech endpoints as fully available, and notes that chat completions do not support logprobs. Tool calling is supported on chat completions. That is exactly the kind of claim this part’s lab probes rather than assumes: run the feature probe against it and record what your version does.

The NPU question

Every Ryzen AI machine is sold with a neural processing unit, and the reasonable question is whether it runs the models in this course. The honest answer, from AMD’s own documentation read on 2026-09-09, is that it runs a documented subset under documented conditions, and that those conditions do not match this course’s inference labs.

What Ryzen AI Software is. AMD describes it as the tools and runtime libraries “for optimizing and deploying AI inference on AMD Ryzen™ AI powered PCs”. The documentation set covers version 1.8.0.

What it requires. The installation page states Windows 11 with build 22621.3527 or newer, an NPU driver of 32.0.203.280 or newer, Visual Studio 2022, CMake 3.26 or newer and a Python distribution. The production driver named on that page, 32.0.203.376, covers Phoenix, Hawk Point, Strix, Strix Halo and Krackan Point processors, so the driver does name this chip.

What it runs. The LLM flow overview documents four execution modes: NPU-only through ONNX Runtime GenAI, a hybrid mode that partitions work between the NPU and the integrated GPU through the same runtime, a GPU mode through llama.cpp, and CPU inference. It lists the supported model families as “Llama-2, Llama-3, Mistral, DeepSeek Distill models, Qwen-2, Qwen-2.5, Qwen-3, Gemma-2, Phi-3, Phi-3.5, and Phi-4”, and says fine-tuned versions of those are supported when prepared following the ONNX Runtime GenAI model guidelines. It lists the NPU-capable processors as “Ryzen AI 300 (STX/KRK)”, with Ryzen AI 7000 and 8000 series limited to GPU and CPU modes.

Three consequences.

The NPU path is Windows-first. The installation page’s operating-system requirement is Windows 11 with a minimum build. Lemonade’s own hardware table lists the XDNA 2 NPU under both Windows and Linux, so the picture is moving; the vendor’s own installation instructions are the conservative reading, and they say Windows.

The model has to be prepared for it. An NPU model is not a GGUF file with a different flag. It is a checkpoint converted for ONNX Runtime GenAI, which is why Lemonade’s NPU models carry distinct names such as Qwen3-4B-Hybrid rather than being the same file you serve to the GPU.

The course’s inference labs use the GPU. Every hands-on page in this course, on every track, runs on the GPU. That is a deliberate decision, not an oversight: the GPU path is the one that is the same shape on all four tracks, runs the same model files, and can be compared. If you want to try the NPU, Lemonade is the least painful route in, and the thing to compare is not only speed but which models you can actually get onto it.

?Reality check

The claim worth examining: “the NPU makes local LLMs fast on this machine”. What the documentation supports is narrower. The NPU has a documented list of model families, a separate model format, a runtime of its own, and an installation path that AMD documents on Windows. It is designed for sustained low-power inference rather than for the largest model that fits in memory, which is what this course spends most of its time on. Treat any comparison you read that does not name the model, the format and the mode as untested.

Windows or Linux for the native path

Both work, and they are good at different things.

Windows is where the NPU stack is documented, where the Ryzen AI installer and the MSI live, and where the GPU-visible memory cap has a published figure. It is also where the rest of this course’s tooling is least at home, which is why the course’s primary Windows path elsewhere is WSL2.

Linux is where ROCm, the HIP build, PyTorch and every script in this course are most comfortable, and where the GPU memory cap is something you raise yourself rather than accept. It is not, on today’s documentation, where the NPU story is.

A reasonable arrangement on a machine you own is Linux for the work in this course and a Windows installation kept for the NPU experiments, rather than trying to make one operating system do both.

Track S — NVIDIA DGX SparkNot supported

This lesson is about the AMD stack; Track S's native path is the previous lesson.

Read the documentation table and the technique that goes with it. NVIDIA’s pages disagree with each other too, as the container tags in the previous lesson showed.

Track X — AMD Ryzen AI Max+ 395

Your primary path. Build the HIP backend into its own directory, keep the Vulkan build from Part 6, install Lemonade, and confirm you can select the backend explicitly. All three are inputs to this part’s lab.

Track M — Apple siliconNot supported

ROCm and Ryzen AI target AMD hardware.

The MLX lesson is your equivalent. The parallel worth noticing is that both vendors ship a framework that assumes their own memory model, and both leave the portable engine as the baseline.

Track N — NVIDIA desktop or laptopNot supported

ROCm and Ryzen AI target AMD hardware.

Read the documentation-status table. The habit of recording which page, which version and which date is the transferable part, and Part 9’s vLLM install lesson will ask for it again.

Match the workload to the actual execution device

An application can expose one chat interface while delegating different operations to a CPU, GPU or NPU. Those devices have distinct supported operators, memory paths and software packages. The presence of an NPU in a machine does not imply that an arbitrary transformer checkpoint can run there unchanged.

Identify the backend chosen by the service and inspect its model-support requirements before downloading conversions. First verify a documented small workload, then change one variable at a time. If a specialised representation is required, record its source and conversion process separately from the ordinary checkpoint.

For measurement, distinguish accelerator utilisation from useful application throughput. A busy device might be executing preprocessing, while generation happens elsewhere. Conversely, a low utilisation sample can miss short bursts. Combine startup logs, sustained measurements and application latency. The lesson’s practical decision is which supported path meets your task and maintenance needs on your installation; avoid treating the names ROCm, Vulkan and NPU as interchangeable switches for the same operation.

Recap

On a Ryzen AI Max+ 395 the compute layer is a choice, not a stack: Vulkan from distribution packages, or ROCm and HIP from AMD. The ROCm compatibility matrix dated 2026-08-14 names the Radeon 8060S and gfx1151 without a support-tier qualifier, while the Linux system-requirements page read on the same day documents an older version and does not mention the chip, so record which page you used and when. The HIP build of llama.cpp needs -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 and belongs in its own build directory so it can be compared with the Vulkan one. Lemonade Server wraps llama.cpp, NPU runtimes and others behind one OpenAI-compatible port, 13305 by default, with --llamacpp rocm or --llamacpp vulkan selecting the backend. The NPU runs a documented list of model families through ONNX Runtime GenAI in NPU-only or hybrid mode, needs models converted for it, and is documented for installation on Windows 11; this course’s inference labs use the GPU on every track.

Check your understanding

Question 1. Two AMD pages read on the same day disagree about whether gfx1151 is supported. What does the course do?

• Pick the more optimistic page
• Record both, with the version each documents and the date each was read, and tell the reader to re-check
• Leave the chip out of the course
• Assume the newer version number is correct

Show the answer and why

Answer: Record both, with the version each documents and the date each was read, and tell the reader to re-check

A vendor stack in motion is the normal case, not an exception. The value of writing down page, version and date is that it turns a future failure into a diagnosis rather than a mystery.

Question 2. Why does the HIP build go into ~/llama.cpp/build-hip rather than the usual build directory?

• HIP requires that name
• So that the Vulkan build from Part 6 survives, and both can be served and measured against each other on the same machine
• To save disk space
• Because cmake refuses to reuse a directory

Show the answer and why

Answer: So that the Vulkan build from Part 6 survives, and both can be served and measured against each other on the same machine

Which of the two backends is faster on this chip is a measurement, and it is one of the more interesting numbers a Track X reader can produce because it is not published anywhere reliable. Keeping both builds is what makes it possible.

Question 3. Which of these are documented requirements or properties of the Ryzen AI NPU path? Select all that apply.

• The installation page states Windows 11 with a minimum build number
• Models must be prepared for ONNX Runtime GenAI rather than served as GGUF
• A documented list of model families is supported, including Llama-3, Qwen-3 and Phi-4
• Any GGUF model can be sent to the NPU with a command-line flag

Show the answer and why

Answer: The installation page states Windows 11 with a minimum build number, Models must be prepared for ONNX Runtime GenAI rather than served as GGUF, A documented list of model families is supported, including Llama-3, Qwen-3 and Phi-4

The last option is the common misunderstanding. The NPU has its own runtime and its own model format, which is why Lemonade lists NPU models under distinct names rather than as backends for the files you already have.

Question 4. Lemonade’s API documentation marks /v1/chat/completions as partially available. How should that affect how you use it?

• Avoid the endpoint entirely
• Test the specific fields your client depends on against your installed version, rather than assuming full OpenAI behaviour
• Use /v1/completions for chat instead
• It only affects image endpoints

Show the answer and why

Answer: Test the specific fields your client depends on against your installed version, rather than assuming full OpenAI behaviour

"OpenAI-compatible" is a spectrum, and the differences show up in the fields you rely on rather than in whether a request succeeds at all. That is why this part’s lab runs a feature probe against every server instead of trusting the label.

Check my answersClear