Qwen3.8 Gets Agent-Trained Quantization That Stops Corrupting File Paths
A community quant of an abliterated Qwen3.8-27B tuned for offensive-security agents, calibrated on real tool-call traffic instead of Wikipedia prose.
Takeaways
- GGUF quant of Qwen3.8-27B-Uncensored-Cyber, calibrated on real agent traffic instead of wiki text.
- 15 GB IQ4_XS file runs on a single V100 with a 262K context and MTP head.
- Calibration corpus: 42 sessions, 648 turns, 528 real tool calls, 1,191 chat-template markers.
- Only 0.42% perplexity gap versus Q5_K_M, and 1.20x faster decode on V100.
- Hits 7/7 on verbatim path copying, the failure mode generic imatrix quants tend to break on.
- Apache-2.0, abliterated cyber-security specialization, not appropriate for user-facing production.
A new 15.0 GiB IQ4_XS quantization of philbert440’s Qwen3.8-27B-Uncensored-Cyber uses an importance matrix built from real agent sessions. The uploader reports exact reproduction of paths and commit hashes in seven functional checks, support for the model’s 262K context setting, and operation on a single V100 through FastLLM.
| Item | Details |
|---|---|
| Source | Upstream checkpoint |
| Download | GGUF repository |
| Primary build | IQ4_XS, 15.0 GiB, 4.25 bits per weight |
| License | Apache-2.0 |
| Uploader’s changes | Quantization and agent-specific calibration; no fine-tuning |
| Intended workload | Local coding agents, security tools, and workflows requiring exact literals |
The source checkpoint is a de-refusal, or “abliterated,” model specialized for offensive security. Its refusal behavior was modified upstream. The GGUF uploader inherited that behavior and made no additional behavioral changes.
Agent traces steer the quantizer
Quantization maps high-precision model weights into lower-precision values, reducing storage and memory bandwidth at the cost of numerical accuracy. A llama.cpp importance matrix records activation statistics from calibration inputs, helping the quantizer preserve the tensor channels that contribute most to those inputs.
Common calibration corpora rely on Wikipedia excerpts or general English prose. Agent sessions contain structures that appear less often in those datasets:
- Tool-call JSON and argument schemas
- Chat-template tokens such as
<|im_start|>and<tool_call> - Repository paths, package names, commit hashes, and device UUIDs
- Mixed Chinese and English content
- Long strings that must be copied without normalization or correction
Sparse representation of those structures gives the quantizer less evidence about the activations involved in reproducing them. The uploader cites one failure in which an agent changed /home/ezra/Documents/Proto-UI to /home/eze/Documents/PotouI, read the corrupted path in a later turn, and treated the mismatch as a context error.