Skip to main content
Back

Qwen3.8 Shrinks to 1.27 GB and Runs Uncensored on Any CPU

A 1.27 GB Q4_K_M build of Qwen3.8-2B with its refusal direction surgically removed, aimed at CPU-only laptops and phones.

  • Qwen3.8-2B Uncensored GGUF: 1.27 GB Q4_K_M build runs on CPU-only laptops and phones.
  • Base is Qwen3.5-2B distilled from a 2.4T A95B teacher by empero-ai.
  • Refusals removed with Heretic v1.4.0 directional ablation, no retraining required.
  • Needs llama.cpp b1-0379a19+ for Qwen3.5 Gated DeltaNet architecture support.
  • Reasoning model, 31.7 tokens/second on Apple Silicon CPU, Apache-2.0 license.
  • Flagged research/personal use only; it will not refuse harmful requests.

The model repository for Qwen3.8-2B-Uncensored-GGUF has recorded thousands of downloads on Hugging Face. It packages a 2 billion-parameter reasoning model as a 1.27 GB GGUF file and modifies its weights to suppress safety refusals. The compact release can run through compatible local inference tools on CPU-based laptops, some Raspberry Pi-class boards, and supported phones using PocketPal. A GPU is optional.

The release uses Q4_K_M quantization, which stores most weights at roughly four-bit precision to reduce disk and memory requirements at some cost to output quality. Its source checkpoint, empero-ai/Qwen3.8-2B, is described as a distillation of a 2.4 trillion-parameter mixture-of-experts teacher into the Qwen3.5-2B student architecture. Distillation trains a smaller model to reproduce behavior learned from a larger one.

The uploader processed that checkpoint with Heretic v1.4.0, an open-source weight-editing tool that reduces refusal behavior without gradient-based retraining. The repository offers an ungated download and lists the Apache-2.0 license. Its model card recommends research and personal use, so organizations should review the base model’s terms and their own deployment requirements before distribution.