---
title: "LocalAI Packs Self-Speculative Decoding Into One Carnice-Qwen3.6 File"
subtitle: "A community contributor packaged samuelcardillo's 35B mixture-of-experts model into GGUFs that bundle a multi-token prediction head, unlocking self-speculative decoding from a single file."
url: "https://alphasignal.ai/news/localai-packs-self-speculative-decoding-into-one-carnice-qwen3-6-file"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-05-17T09:12:50.000Z"
updated: "2026-08-22T02:02:11.261Z"
type: "Model"
topics:
  - "Gpus"
  - "Llms"
subtopics:
  - "Quantization"
  - "Mixture Of Experts"
upvotes: 2518
source_url: "https://huggingface.co/mudler/Carnice-Qwen3.6-MoE-35B-A3B-APEX-MTP-GGUF"
comments: 0
truncated: true
---

# LocalAI Packs Self-Speculative Decoding Into One Carnice-Qwen3.6 File

> A community contributor packaged samuelcardillo's 35B mixture-of-experts model into GGUFs that bundle a multi-token prediction head, unlocking self-speculative decoding from a single file.

Speculative decoding usually asks you to run two models side by side: a big target and a smaller draft that guesses ahead. A new set of quantized GGUFs on Hugging Face collapses that setup into one file. Community contributor [mudler](https://huggingface.co/mudler), working under the [LocalAI](https://github.com/mudler/LocalAI) project's APEX quantization effort, has published [Carnice-Qwen3.6-MoE-35B-A3B-APEX-MTP-GGUF](https://huggingface.co/mudler/Carnice-Qwen3.6-MoE-35B-A3B-APEX-MTP-GGUF), a quantized build of [samuelcardillo/Carnice-Qwen3.6-MoE-35B-A3B](https://huggingface.co/samuelcardillo/Carnice-Qwen3.6-MoE-35B-A3B) that ships the model's multi-token prediction head inside the same GGUF as the trunk.

## One file, no draft model required

Multi-token prediction, or MTP, is a small extra transformer block trained to guess the next token cheaply. During inference the main model verifies that guess, and when the draft is correct you effectively get two tokens for the cost of one forward pass. Traditionally in llama.cpp you had to point `--model-draft` at a completely separate smaller model, matching tokenizers and juggling extra memory.

Thanks to [llama.cpp PR #22673](https://github.com/ggml-org/llama.cpp/pull/22673), the MTP head can now live in the same file as the main weights. With a recent llama.cpp (commit 255582687 or newer) you can enable self-speculative decoding using a single file, with no separate draft model:

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [mudler/Carnice-Qwen3.6-MoE-35B-A3B-APEX-MTP-GGUF](https://huggingface.co/mudler/Carnice-Qwen3.6-MoE-35B-A3B-APEX-MTP-GGUF)
- [mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF](https://huggingface.co/mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF)
- [Speculative Decoding | ggml-org/llama.cpp DeepWiki](https://deepwiki.com/ggml-org/llama.cpp/8.3-speculative-decoding)
- [APEX Quant project](https://github.com/mudler/apex-quant)
- [llama.cpp speculative decoding docs](https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md)