--- title: "Google Shrinks Gemma 4 to Under 1GB Without Losing Reasoning Power" subtitle: "Google releases QAT-optimized Gemma 4 checkpoints that cut memory 40–50% while preserving near-full model quality, enabling local inference on phones, laptops, and consumer GPUs." url: "https://alphasignal.ai/news/google-shrinks-gemma-4-to-under-1gb-without-losing-reasoning-power" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "Google for Developers" company_url: "https://alphasignal.ai/company/googledevs" published: "2026-06-05T16:13:20.000Z" updated: "2026-06-16T20:16:25.582Z" type: "News" topics: - "Llms" - "Gpus" subtopics: - "Small Models" - "Quantization" - "Edge Deployment" upvotes: 1348 read_time: "6 min" source_url: "https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/" comments: 0 --- # Google Shrinks Gemma 4 to Under 1GB Without Losing Reasoning Power > Google releases QAT-optimized Gemma 4 checkpoints that cut memory 40–50% while preserving near-full model quality, enabling local inference on phones, laptops, and consumer GPUs. Google just shipped [Quantization-Aware Training (QAT) checkpoints](https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/) for the entire Gemma 4 family, and this is one of the most practically useful open-weights releases of the year. The goal: run capable AI models on the hardware you already own, without the quality penalty that normally comes with aggressive compression. The new checkpoints are optimized with QAT to make Gemma 4 even more efficient, enabling local inference on everyday edge devices and consumer GPUs, by simulating quantization during training to minimize quality loss when the model is compressed. The result is a lineup of models that spans from a sub-1GB phone-ready variant all the way to a 31B dense model that now fits on a laptop. ## Why QAT beats the usual approach To understand why this matters, you need to know the difference between two compression strategies. The standard approach, Post-Training Quantization (PTQ), takes a fully trained model and converts its weights to a lower-precision format after the fact. It is fast and cheap, but the model was never designed for that precision. PTQ takes a fully trained model and shoves its weights into a lower-precision format after the fact. The result is smaller and faster, but accuracy often degrades because the model never learned to compensate for the quantization noise. QAT flips this entirely: QAT simulates low-precision math during the forward pass, so the model learns to compensate for quantization error while it is still training. The payoff is significant. QAT delivers roughly 40-50% lower memory usage, and quality remains within a few percentage points of the FP16 reference on standard benchmarks like MMLU and GSM8K. Critically, accuracy on INT4 QAT is significantly higher than INT4 PTQ, despite similar resource usage. For Gemma 4 specifically, the architecture has a few traits that make PTQ painful: the interleaved local/global attention and the 256K context window can amplify rounding errors. QAT was the right tool for this model. ## A novel schema built for mobile The most technically interesting part of this release is not the Q4\_0 desktop variant, it is the custom mobile quantization schema Google engineered from scratch for the E2B and E4B edge models. Standard compression formats are often hard for mobile processors to run efficiently, so Google rethought the problem entirely. The mobile schema combines four techniques: - **Static activations:** Instead of computing scaling factors on the fly during inference, these are pre-calculated during training. This reduces workload on mobile chips and makes responses faster. - **Channel-wise quantization:** Data is structured to fit the design of mobile accelerators, allowing the hardware to run calculations natively without slow workarounds. - **Targeted 2-bit quantization:** The token-generation layers are compressed to 2-bit, while the core reasoning layers stay at higher precision. This saves storage without hurting reasoning quality. - **Embedding and KV cache optimization:** Compression is focused on the vocabulary list and the model's short-term memory (KV cache), drastically reducing active memory during long conversations. This release includes QAT checkpoints for the popular Q4\_0 quantization format as well as a novel quantization format specialized for mobile use cases. The text-only Gemma 4 E2B model, without audio and vision encoders, requires less than 1 GB of memory. ![Comparison table showing model sizes across different quantization formats and mobile variants for Gemma 4 language models](https://storage.googleapis.com/gweb-uniblog-publish-prod/images/2096x1182-D.width-1000.format-webp.webp) ## The full model lineup The Gemma 4 models optimized with QAT are available in five sizes: Gemma 4 E2B, Gemma 4 E4B, Gemma 4 12B, Gemma 4 26B A4B, and Gemma 4 31B. Each targets a different hardware tier: - **E2B / E4B:** Phones, Raspberry Pi, and other edge devices. The mobile QAT schema applies here. - **12B:** Laptops and mid-range consumer GPUs. - **26B A4B (MoE):** High-throughput workstations. The 26B-A4B now runs locally on consumer GPUs with as little as 16GB of RAM. - **31B:** The larger 26B-A4B and 31B QAT models now work on laptops rather than just strong home GPUs. ## What is it actually good for? The models are well-suited for reasoning, agentic workflows, coding, and multimodal understanding. In practice, the QAT advantage shows up most clearly in two scenarios: coding tasks and long-context work. The QAT version holds a small but consistent edge on HumanEval-style prompts, roughly 2-3 percentage points, and the bigger win is stability: long-context summarization at 128K tokens does not drift the way some PTQ models do. One important caveat: hardware fragmentation still complicates support matrices across mobile inference and laptop AI fleets. Not every mobile SoC will get the same performance, and some hardware-specific tuning may still be required for production deployments. ## What just became possible The real shift here is not just smaller models. It is that the quality bar for local inference has been raised. The 26B parameter Gemma 4 now runs with the memory footprint of a 14B model, but with the reasoning capabilities of a 31B dense architecture. That is a meaningful jump for anyone building offline-capable applications. Local inference offers several benefits, including lower latency, improved privacy, reduced cloud costs, and offline functionality. For mobile app developers, on-device AI assistants, and anyone building in bandwidth-constrained environments, these checkpoints open up use cases that were simply impractical before. This also fits a broader industry pattern. The release highlights a growing trend across the AI industry: moving intelligence closer to the user. Google is not alone in this push, but the combination of QAT quality, a custom mobile schema, and broad ecosystem support makes this release stand out. ## Getting started right now The weights are live and the ecosystem integrations are already in place. Here is where to go: - **[Hugging Face](https://huggingface.co/collections/google/gemma-4-qat-q4-0):** Q4\_0 GGUF files and compressed tensor formats for vLLM are ready to download. - **llama.cpp / Ollama / LM Studio:** Drop-in support, no extra configuration needed. - **MLX:** Apple Silicon optimized variants available. - **vLLM / SGLang:** For serving larger models at scale. - **LiteRT-LM:** Google's lightweight runtime for optimized edge deployment. - **Unsloth:** Fine-tune the QAT weights directly. - **Transformers.js:** Run the models in-browser via ONNX. The models are free to use under the Gemma Terms of Use. If you want to go deeper, the [official documentation](https://ai.google.dev/gemma/docs/core#qat) covers deployment options for each format and hardware target. The sub-1GB E2B model running fully offline on a phone is a concrete demonstration that the gap between frontier-model quality and edge-device constraints is closing faster than most expected. ## Sources - [X post by @googledevs](https://x.com/googledevs/status/2062930781945700861) - [Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency](https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/) - [Gemma 4 QAT Q4_0 Collection - Hugging Face](https://huggingface.co/collections/google/gemma-4-qat-q4-0) - [Unsloth Gemma 4 QAT Documentation](https://unsloth.ai/docs/models/gemma-4/qat) - [Gemma 4 Goes Mobile: What Google's New QAT Checkpoints Mean for On-Device AI](https://dev.to/lymy1205/gemma-4-goes-mobile-what-googles-new-qat-checkpoints-mean-for-on-device-ai-551f) - [google/gemma-4-12B-it-qat-q4_0-unquantized - Hugging Face](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-unquantized)