TurinTech Shrinks Google's Gemma 4 E4B to 4 GB With 98.4% Accuracy
TurinTech's Artemis conversion shrinks Google's Gemma 4 E4B from 14.9 GB down to 4.22 GB while matching the full-precision model's next token 98.4% of the time.
Takeaways
- TurinTech released Artemis GGUF conversions of Google's Gemma 4 E4B on Hugging Face.
- Default Q4_0 file is 4.22 GB, a 72% cut from the 14.9 GB BF16 original.
- Next-token agreement with BF16 reaches 98.4% for Q4_0 and 94.7% for the 3.84 GB IQ3_M.
- Built from Google's quantization-aware training checkpoint, runs on stock llama.cpp, Ollama, LM Studio and vLLM.
- Roughly 45 tokens/s generation measured on an Apple M3 Max with llama.cpp Metal.
- Apache-2.0 licensed, text only, text generation and tool use supported.
TurinTech has released three GGUF conversions of Google’s Gemma 4 E4B instruction model for local inference. The main Q4_0 file occupies 4.22 GB, down from 14.9 GB for the BF16 reference. In TurinTech’s evaluation, it selected the same next token as the reference at 98.4% of 640 scored positions. That figure measures token-level fidelity on a compact test; broader quality depends on the task, prompt and generation length.
QAT softens the 4-bit cut
TurinTech produced the conversions with Artemis, its code-optimisation platform, using Google’s quantisation-aware training checkpoint. QAT exposes a model to simulated low-precision arithmetic during training, allowing its weights to adapt before conversion. BF16 stores values with 16 bits, while these quantisations store most weights at roughly three or four bits plus scaling data. The resulting files use standard GGUF and run in stock llama.cpp without custom kernels.
Three sizes, measured trade-offs
All three variants derive from the same QAT checkpoint, with progressively smaller files and greater reported divergence from the BF16 model.
| Variant | Size | Size reduction | Reported checks |
|---|---|---|---|
Q4_0 |
4.22 GB | 72% | 98.4% next-token agreement across 640 fixed-context positions; 93.9% agreement across greedy 32-token continuations |