Skip to main content
Back

Moonshot's Kimi K3 Coding Skills Distilled Into a 5.9 GB Local Model

A community-built 9B Qwen3.5 fine-tune, trained on verified Kimi K3 agent traces and packaged as GGUF, now runs on a laptop via llama.cpp.

Model, Full Access
  • GGUF quants of Qwen3.5-9B fine-tuned on Kimi K3 agent and coding traces, on Hugging Face
  • Recommended Q4_K_M file is 5.9 GB and runs on a laptop via llama.cpp
  • Trained on 3,956 verified next-step rows from 582 real Kimi K3 sessions
  • Task mix is 52.9% building, 25.6% debugging, 13.6% tool calling
  • Apache 2.0 license, 9k+ downloads, works with Ollama, LM Studio, Jan
  • Specialist for local coding agents, not a general-purpose frontier replacement

A community release called Qwen K3 distillation combines a 9-billion-parameter Qwen3.5 checkpoint with supervised fine-tuning on coding and tool-use traces generated by Moonshot’s Kimi K3. The recommended Q4_K_M build occupies about 5.9 GB, making the model practical for local coding agents and OpenAI-compatible development tools on many consumer systems.

The repository has recorded more than 9,000 downloads despite limited promotion. That count shows active experimentation, while the absence of published comparative benchmarks leaves coding quality, tool-call accuracy, and regression rates unmeasured.

Base model
Qwen3.5 9B
Training method
Supervised fine-tuning on Kimi K3 session traces
Training set
3,956 cumulative examples from 582 accepted trajectories
Primary focus
Coding, debugging, tool use, and multi-step agent workflows
Distribution format
GGUF, with static and importance-matrix variants
Declared license
Apache-2.0

Qwen weights learn Kimi’s workflow

The upload contains static GGUF quantizations of khazarai/Qwen3.5-9B-Kimi-k3-Distilled, plus a separate importance-matrix variant. The checkpoint retains the Qwen3.5 architecture while learning response patterns from the Kimi trace dataset.

The release uses behavioral distillation through supervised fine-tuning on teacher-generated sessions. Its documentation describes training on Kimi’s messages and tool interactions rather than matching the teacher model’s internal logits. Quantization happens after fine-tuning and reduces storage and memory requirements by representing weights at lower precision.

The repository also includes an mmproj projector file for multimodal use. Vision input requires a compatible llama.cpp build, the matching projector, and an appropriate prompt template. The model card provides no multimodal evaluation results.