---
title: "Unsloth Makes DeepSeek-V4-Flash Finally Runnable Locally on 160 GB"
subtitle: "Unsloth ships lossless GGUF builds of DeepSeek-V4-Flash, making the 284B MoE model locally runnable on 110GB RAM after a critical llama.cpp fix landed July 7th."
url: "https://alphasignal.ai/news/unsloth-makes-deepseek-v4-flash-finally-runnable-locally-on-160-gb"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Unsloth AI"
company_url: "https://alphasignal.ai/company/unslothai"
published: "2026-07-06T10:21:51.000Z"
updated: "2026-07-09T22:27:48.796Z"
type: "Model"
topics:
  - "Llms"
  - "Open Source"
subtopics:
  - "Long Context"
  - "Mixture Of Experts"
upvotes: 3191
source_url: "https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF"
comments: 0
truncated: true
---

# Unsloth Makes DeepSeek-V4-Flash Finally Runnable Locally on 160 GB

> Unsloth ships lossless GGUF builds of DeepSeek-V4-Flash, making the 284B MoE model locally runnable on 110GB RAM after a critical llama.cpp fix landed July 7th.

DeepSeek's V4 family arrived in April with a genuinely new architecture, and the smaller of its two models, [DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash), has been one of the most-watched open-weight releases of 2026\. Now, Unsloth has published a properly quantized GGUF version that finally makes local inference practical, following a key llama.cpp bug fix that landed on July 7th.

## What got released, and why it took a while

DeepSeek-V4-Flash is a Mixture-of-Experts model with 284B total parameters and only 13B activated per token, supporting a context length of one million tokens. The open weights dropped under the MIT license, but running them locally was a mess for months. llama.cpp added DeepSeek V4 support, but multi-turn conversations would not function well when using KV cache quantization , it has since been fixed and merged in llama.cpp since July 7th, 2026\. Before that fix, the model would produce garbled output mid-conversation.

You can now run DeepSeek-V4-Flash in Unsloth Studio with toggles for High and Max thinking. The [unsloth/DeepSeek-V4-Flash-GGUF](https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF) repo is the first set of GGUF builds that Unsloth considers production-ready for this model.

## The quantization problem is unusual here

V4-Flash is not a standard BF16 model that you compress down. DeepSeek-V4-Flash is quantization-aware-trained: the official checkpoint stores its routed experts , 96% of the model , natively in MXFP4, and everything else in FP8 or BF16\. GGUF's MXFP4 is exactly that format, so Unsloth repacks the experts bit-for-bit, and FP8 dequantizes into BF16 with no rounding. 

This matters because other GGUF conversions got it wrong. Unsloth checked every tensor against the official DeepSeek weights: all 1,328 are bit-identical, and it stays lossless at inference with KL-divergence \~0 and 100% top-token agreement. Non-Unsloth DeepSeek-V4-Flash GGUFs were converted without these paths, thus deviating from the official weights. The practical consequence: other community GGUFs re-quantize the already-FP4 experts to Q4\_K or IQ2\_XXS, introducing 5–30% weight error per layer. Unsloth's builds keep those expert weights untouched.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [unsloth/DeepSeek-V4-Flash-GGUF — Hugging Face](https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF)
- [DeepSeek-V4: How to Run Locally — Unsloth Documentation](https://unsloth.ai/docs/models/deepseek-v4)
- [deepseek-ai/DeepSeek-V4-Flash — Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash)
- [DeepSeek V4 Flash — OpenRouter](https://openrouter.ai/deepseek/deepseek-v4-flash)
- [DeepSeek V4 in the wild, and how to run it on RunPod](https://www.runpod.io/blog/deepseek-v4-in-the-wild-and-how-to-run-it-on-runpod)