---
title: "Google's gemma.cpp Runs Gemma 3 on CPU With No GPU Required"
subtitle: "Google's gemma.cpp brings Gemma 2 and 3 inference to pure C++ with no GPU needed, targeting researchers who want to hack on model internals"
url: "https://alphasignal.ai/news/google-s-gemma-cpp-runs-gemma-3-on-cpu-with-no-gpu-required"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Google"
company_url: "https://alphasignal.ai/company/google"
published: "2026-06-19T13:38:06.000Z"
updated: "2026-06-20T20:01:59.192Z"
type: "Repo"
topics:
  - "Open Source"
  - "Infra"
subtopics:
  - "Inference Optimization"
upvotes: 3460
source_url: "https://github.com/google/gemma.cpp"
comments: 0
truncated: true
---

# Google's gemma.cpp Runs Gemma 3 on CPU With No GPU Required

> Google's gemma.cpp brings Gemma 2 and 3 inference to pure C++ with no GPU needed, targeting researchers who want to hack on model internals

[gemma.cpp](https://github.com/google/gemma.cpp) is Google's lightweight, standalone C++ inference engine for running Gemma models directly on CPU, with no GPU required. It supports Gemma 2, Gemma 3, and PaliGemma 2, and is built for a specific kind of user: someone who wants to poke around inside the model, not just call it as a black box. Think of it as the spiritual cousin of `llama.c` and `ggml`, but purpose-built for Google's own model family.

## The gap it fills

gemma.cpp targets experimentation and research use cases, and is intended to be straightforward to embed in other projects with minimal dependencies and easily modifiable, with a small \~2K LoC core implementation (along with \~4K LoC of supporting utilities). That's the key pitch. Most C++ inference runtimes are production-hardened and opaque by design. Python ML frameworks, on the other hand, abstract away the low-level compute through compilation. gemma.cpp sits in the middle: readable, hackable, and fast enough to be genuinely useful.

The author, Austin Huang, put it plainly in the Hacker News thread at launch: the weights should be the same across formats, but differences can arise due to quantization and subtle implementation differences, which has been a pain point in the ML ecosystem for a while. The project is explicitly not trying to replace llama.cpp or vLLM for production. It's for the people who want to add sparsity support, custom quantization, or a novel decoding strategy and see it working with a real model in minimal code.

## What's under the hood

The project uses the Google Highway Library to take advantage of portable SIMD for CPU inference. SIMD (Single Instruction, Multiple Data) is a CPU technique that processes multiple data points in a single instruction cycle, which is how you squeeze real throughput out of a CPU for matrix math. Highway is notable because it picks the best available instruction set (AVX2, AVX-512, NEON, etc.) at runtime, so the same binary runs efficiently on any machine without recompilation.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [google/gemma.cpp - GitHub](https://github.com/google/gemma.cpp)
- [Gemma.cpp: lightweight, standalone C++ inference engine for Gemma models | Hacker News](https://news.ycombinator.com/item?id=39481554)
- [google/gemma.cpp | There's An AI For That](https://theresanaiforthat.com/company/google/repository/gemma.cpp/)
- [Run Gemma content generation and inferences | Google AI for Developers](https://ai.google.dev/gemma/docs/run)
- [gemma.cpp/DEVELOPERS.md at main · google/gemma.cpp](https://github.com/google/gemma.cpp/blob/main/DEVELOPERS.md)