Google Runs Gemma 4 E2B Fully Offline on a $175 Raspberry Pi
Google's LiteRT runs Gemma 4 E2B on a $175 Raspberry Pi 5 with voice, vision, and robotics entirely offline at 300 words per minute
- Google's LiteRT runtime now runs Gemma 4 E2B on a Raspberry Pi 5 with voice, vision, and robotics fully offline.
- Gemma 4 E2B achieves 9 tokens/sec decode and 99 tokens/sec prefill in just 1,432 MB of RAM on the Pi 5.
- The efficient tokenizer (~4.2 chars/token) translates 9 tokens/sec into ~300 words per minute, twice normal human speech speed.
- Per-Layer Embeddings (PLE) let E2B carry 5.1B total parameters while only activating 2.3B in RAM, enabling the tiny memory footprint.
- The Reachy Mini demo splits YOLO object detection to the Pi's GPU and Gemma reasoning to the CPU, running both concurrently.
- Install via
pip install litert-cli; models are free and available on the LiteRT Hugging Face Community.
A $175 single-board computer running a full multimodal AI stack, completely offline, with no cloud calls, no API fees, and no latency penalty from a network hop. That is what Google just demonstrated with LiteRT and Gemma 4 E2B on the Raspberry Pi 5, using the Reachy Mini robot as a live showcase. The combination handles voice input, object detection via YOLO, language reasoning, and text-to-speech output, all running in parallel on a single board.
The hardware-software equation that makes this work
The key enabler on the software side is LiteRT, Google AI Edge's on-device inference runtime. Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can deploy Gemma right out of the box, with CPU acceleration via XNNPACK ensuring the model family is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.
On the model side, the star is Gemma 4 E2B. The "E" stands for "effective" parameters, and the name hides a clever trick. The E2B and E4B models use per-layer embeddings (PLE) inherited from Gemma 3n. Their total parameter counts are substantially larger than the numbers in their names: E2B has roughly five billion stored parameters but only 2.3 billion effective parameters. The large embedding tables live in flash storage, not RAM, so they barely touch your working memory budget.
Here is what PLE actually does: standard transformers give each token a single embedding vector at input. PLE adds a parallel lower-dimensional conditioning pathway, producing a small dedicated vector per layer, letting each decoder layer receive token-specific information only when relevant rather than requiring everything to be frontloaded into a single embedding. The practical result is that a 2.3B-active model carries the representational depth of the full 5.1B parameter count while fitting in under 1.5 GB of memory with 2-bit quantization.
What 9 tokens/sec actually means in practice
On a Raspberry Pi 5, LiteRT-LM achieves 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. Nine decode tokens per second sounds modest, but context matters. Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an end-to-end generation speed of ~27.3 characters per second, roughly 300 words per minute, which is twice the speed of normal human speech (~150 wpm). For a voice assistant, that means the model is generating text faster than a human can speak it back.