Skip to main content
Back

mradermacher Ships Huihui-Ornith 35B Uncensored for Local GPU Runners

A 36B Qwen3-MoE with refusals ablated gets a full ladder of GGUF quants, putting an uncensored sparse model on 16GB GPUs.

Model, Full Access
  • mradermacher published static GGUF quants of Huihui-Ornith-1.5-35B-A3B-abliterated, from 13.3GB Q2_K to 37.9GB Q8_0.
  • Base is Ornith-1.5-35B-A3B, a Qwen3.5-MoE with roughly 3B active parameters out of 36B total.
  • Abliteration by huihui-ai touches only layers 11 to 29, leaving edge layers untouched.
  • Repo includes mmproj files for vision input and works with llama.cpp, Ollama, LM Studio, and Jan.
  • A community v2 applies full 40-layer refusal ablation for users who want deeper uncensoring.
  • Independent tests on sibling ICE quants hit 81.2% on an L7 benchmark at 23GB VRAM, a 35B-A3B record.

mradermacher has published static GGUF conversions of Huihui-Ornith-1.5-35B-A3B-abliterated, a Qwen3.5 mixture-of-experts model derived from Ornith-1.5. The GGUF repository spans 13.3 GB Q2_K through 37.9 GB Q8_0, giving local developers options for constrained GPUs, larger workstations, and CPU offload.

File size establishes only the initial memory requirement. The runtime also needs memory for the context cache, compute buffers, and multimodal projector when used. A 13.3 GB model can be tight on a 16 GB GPU at longer contexts, while a 21.8 GB quant may require partial CPU offload on a 24 GB card.

One model, two parameter counts

The source checkpoint uses 35B in its name, while its card reports roughly 36B total parameters. About 3B parameters are active for each token because the architecture routes tokens through a subset of its experts.

A mixture-of-experts model reduces arithmetic by activating selected experts, but the full set of weights must remain in RAM, VRAM, or an offloaded memory tier. Attention, routing, cache operations, and memory bandwidth also affect throughput, so performance will not match a dense 3B model automatically.

Abliteration changes the refusal path

The edited checkpoint applies abliteration, a weight-editing method that estimates an activation direction associated with refusals and removes part of that direction from selected layers. Huihui reports editing layers 11 through 29 rather than every layer.

The limited scope can produce uneven behavior. The model may comply with prompts rejected by the original checkpoint, retain some refusals, or show instruction-following and coherence changes. Abliteration suppresses a learned behavior pattern; it does not guarantee universal compliance or remove every safety-related representation.

A technical guide reports that the 35B model stores its experts in a fused tensor that the editing tool cannot access, leaving the first pass focused on attention components. A separate v2 attributed to jhone888 reportedly applies refusal-direction edits across all 40 layers and verifies the result numerically. That checkpoint should be evaluated separately because it contains broader weight changes.