---
title: "Qwen3.8 Hits 49 Tokens per Second on AMD's Budget APU"
subtitle: "A community quantization pairs a 27B Qwen model with multi-token prediction and AMD's IU4 matrix path, hitting ~49 tokens per second on a single Strix Halo APU."
url: "https://alphasignal.ai/news/qwen3-8-hits-49-tokens-per-second-on-amd-s-budget-apu"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-08-22T20:46:25.000Z"
updated: "2026-08-27T20:04:25.089Z"
type: "Model"
topics:
  - "Gpus"
  - "Llms"
subtopics:
  - "Quantization"
  - "Edge Deployment"
  - "Small Models"
upvotes: 33
source_url: "https://huggingface.co/cafonez/Qwen3.8-27B-ROCmI4-MTP-GGUF"
comments: 0
truncated: true
---

# Qwen3.8 Hits 49 Tokens per Second on AMD's Budget APU

> A community quantization pairs a 27B Qwen model with multi-token prediction and AMD's IU4 matrix path, hitting ~49 tokens per second on a single Strix Halo APU.

A community upload on Hugging Face is drawing attention from people trying to squeeze real throughput out of AMD's Strix Halo APUs. The [Qwen3.8-27B-ROCmI4-MTP-GGUF](https://huggingface.co/cafonez/Qwen3.8-27B-ROCmI4-MTP-GGUF) release bundles a 4-bit quantized 27B model with an embedded speculative decoder, and reports decode speeds around 49 tokens per second on a single Ryzen AI MAX+ 395 with integrated Radeon 8060S graphics.

This is a conversion and quantization of Qwen/Qwen3.8-27B rather than a fresh training run. The main GGUF includes the model's one-layer NextN/MTP head, so no separate draft model is required. The interesting behavior shows up once you feed it into the right runtime.

## Why the ROCmI4 label matters

The file uses a custom GGUF weight type called `Q4_0_ROCMI4`, which stock llama.cpp does not understand. Running it requires the ROCmFPX fork at a specific pinned commit, since neither the custom ROCmI4 type nor the qualified IU4/MTP path exist in mainline builds.

A quick glossary helps here:

* **ROCmI4**: the GGUF weight format with signed four-bit weight codes packed two per byte with block scales.
* **INT4**: the generic four-bit integer width.
* **IU4**: AMD's `v_wmma_i32_16x16x16_iu4` matrix instruction on gfx1151.
* **W4A4**: four-bit weights and four-bit activations during accelerated matrix multiplication.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [cafonez/Qwen3.8-27B-ROCmI4-MTP-GGUF](https://huggingface.co/cafonez/Qwen3.8-27B-ROCmI4-MTP-GGUF)
- [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX)
- [julianmb/q38rocm](https://github.com/julianmb/q38rocm)
- [kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF)