Google Shrinks Gemma 4 to 1GB so It Runs on Your Phone
Google's open-weight family gets quantization-aware checkpoints across every size, shrinking the smallest variant to a 1GB memory footprint while preserving quality.
- Google released Gemma 4 QAT checkpoints for E2B, E4B, 12B, 26B-A4B, and 31B.
- New mobile quantization format shrinks Gemma 4 E2B to a 1GB memory footprint.
- QAT preserves near-bfloat16 quality with roughly 3x less memory, beating standard PTQ.
- Available now in LM Studio as GGUF and MLX, plus vLLM compressed-tensors w4a16.
- 31B model hits 85.2% MMLU Pro, 80% LiveCodeBench v6, 89.2% AIME 2026.
- Shipped under Apache 2.0 with multimodal input, tool use, and 128K-256K context.
Google just dropped a set of quantization-aware training (QAT) checkpoints for every member of the Gemma 4 family, and they're already live in LM Studio. The release covers the entire lineup, from the tiny E2B up to the 31B dense model, and is specifically engineered to let these models squeeze onto consumer GPUs, laptops, and even phones without the usual quality cliff that comes from aggressive compression.
What QAT actually changes
The standard approach for shrinking a model after training is post-training quantization (PTQ), where you take fully trained weights and round them down to lower precision such as 4-bit integers. It works, but standard Post-Training Quantization (PTQ) often leads to performance degradation. QAT takes a different route: by simulating quantization during training, QAT minimizes quality loss when the model is compressed, so the network learns to be robust to the rounding errors instead of being blindsided by them at the end.
The practical payoff is significant. This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model. Unsloth's port advertises roughly 3x less memory use and near original accuracy versus the full-precision checkpoints.
Two formats, one of them new
This release includes QAT checkpoints for the popular Q4_0 quantization format as well as a novel quantization format specialized for mobile use cases. The mobile format is the headline-grabber: Using this mobile format, we've reduced the memory footprint of Gemma 4 E2B to 1GB. Together, these dramatically reduce memory requirements while preserving the capabilities and quality you expect from Gemma 4.
For server-side workloads, there's also a vLLM-friendly variant. Compressed Tensors (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B, and 31B.
Which model goes where
The Gemma 4 family covers a wide deployment spectrum, and the QAT versions inherit the same capabilities. Memory requirements in LM Studio range from roughly 4GB for E2B up to 19GB for the 31B dense model, with the 26B-A4B mixture-of-experts sitting at around 15.6GB. All of them retain tool use, vision input, and reasoning, and are distributed in both GGUF and MLX.
| Model | Type | Context | Modalities |
|---|---|---|---|
| E2B | Dense (2.3B effective) | 128K | Text, Image, Audio |
| E4B | Dense (4.5B effective) | 128K | Text, Image, Audio |
| 12B | Dense | 128K | Text, Image, Audio |
| 26B-A4B | MoE (3.8B active) | 256K | Text, Image |
| 31B | Dense | 256K | Text, Image |
The 26B-A4B is the interesting middle child. It contains 25.2B total parameters but only routes through 3.8B at inference, so throughput is closer to a 4B dense model than a 26B one. The 12B slot is also new , Google recently shipped it to bridge the gap between our E4B and 26B MOE models.
How to actually run it
The fastest path is LM Studio, where the QAT checkpoints show up directly under the google/gemma-4 namespace. For a more programmatic workflow, the unquantized QAT weights are on Hugging Face under names like google/gemma-4-E2B-it-qat-q4_0-unquantized, ready to be fed through llama.cpp, MLX, or vLLM. Everything ships under Apache 2.0, so commercial use is unrestricted.
Where it shines, and where it stumbles
On benchmarks, Gemma 4 31B posts 85.2% on MMLU Pro, 80.0% on LiveCodeBench v6, and 89.2% on AIME 2026 with no tools, putting it in serious frontier-adjacent territory for an open model. The MoE 26B-A4B is close behind, hitting 82.6% MMLU Pro at a fraction of the active compute. Vision is also strong: the 31B scores 76.9% on MMMU Pro and 85.6% on MATH-Vision.
The community reception of the base Gemma 4 release has been mixed on raw speed. Early testers reported numbers like 11 tokens/sec on Gemma 4 26B-A4B vs 60+ tokens/sec on Qwen 3.5 35B-A3B on the same 5060 Ti 16GB, along with higher VRAM usage at equivalent quantization levels. Several reviewers explicitly called out that we need QAT quantizations before the smaller models can truly compete on efficiency , which is exactly the gap today's release is trying to close.
Why this matters beyond the headline
The broader story is that QAT is becoming table stakes for serious open-weight releases. Google followed the same playbook with Gemma 3, where QAT made the 27B variant runnable on a single RTX 3090. With Gemma 4, the same trick now stretches all the way down to a phone-friendly 1GB binary for E2B and, on the other end, makes the 31B dense model viable on a workstation GPU.
For anyone building local-first applications, on-device agents, or privacy-sensitive pipelines, the calculus has shifted again: a multimodal, tool-using, reasoning-capable model with native function calling and a 128K-256K context now fits inside hardware budgets that were unthinkable a year ago. The trade-off between running locally and running well is getting noticeably thinner.
Comments
