Google Runs Gemma 4 E2B Fully Offline on a $175 Raspberry Pi

A $175 single-board computer running a full multimodal AI stack, completely offline, with no cloud calls, no API fees, and no latency penalty from a network hop. That is what Google just demonstrated with LiteRT and Gemma 4 E2B on the Raspberry Pi 5, using the Reachy Mini robot as a live showcase. The combination handles voice input, object detection via YOLO, language reasoning, and text-to-speech output, all running in parallel on a single board.

The hardware-software equation that makes this work

The key enabler on the software side is LiteRT, Google AI Edge's on-device inference runtime. Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can deploy Gemma right out of the box, with CPU acceleration via XNNPACK ensuring the model family is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.

On the model side, the star is Gemma 4 E2B. The "E" stands for "effective" parameters, and the name hides a clever trick. The E2B and E4B models use per-layer embeddings (PLE) inherited from Gemma 3n. Their total parameter counts are substantially larger than the numbers in their names: E2B has roughly five billion stored parameters but only 2.3 billion effective parameters. The large embedding tables live in flash storage, not RAM, so they barely touch your working memory budget.

Here is what PLE actually does: standard transformers give each token a single embedding vector at input. PLE adds a parallel lower-dimensional conditioning pathway, producing a small dedicated vector per layer, letting each decoder layer receive token-specific information only when relevant rather than requiring everything to be frontloaded into a single embedding. The practical result is that a 2.3B-active model carries the representational depth of the full 5.1B parameter count while fitting in under 1.5 GB of memory with 2-bit quantization.

What 9 tokens/sec actually means in practice

On a Raspberry Pi 5, LiteRT-LM achieves 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. Nine decode tokens per second sounds modest, but context matters. Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an end-to-end generation speed of ~27.3 characters per second, roughly 300 words per minute, which is twice the speed of normal human speech (~150 wpm). For a voice assistant, that means the model is generating text faster than a human can speak it back.

The Reachy Mini pipeline: a blueprint for edge agents

The Reachy Mini pipeline is a powerful showcase of low-latency, real-time edge AI inference running entirely on the Raspberry Pi 5. By leveraging LiteRT, the system splits intensive vision and language workloads into a concurrent, dual-processing architecture across CPU and GPU. The split is deliberate and smart:

• GPU lane: Camera frames are streamed to the Pi, where Ultralytics YOLO detection runs continuously on the GPU, avoiding resource contention and freeing up the CPU.
• CPU lane (ASR): When the user speaks, the Moonshine ASR component transcribes the audio into text directly on the CPU.
• CPU lane (LLM): The Gemma 4 E2B model processes the resulting transcript alongside the latest visual metadata to generate low-latency, streaming responses, such as speech replies and physical robotic gestures.
• TTS: A text-to-speech component synthesizes the generated text into audio in streaming, sending the voice back to the robot.

The GPU here is the Raspberry Pi 5's integrated Broadcom VideoCore VII, enabled through LiteRT's WebGPU (Vulkan) backend via ML Drift. This is not a discrete AI accelerator. It is the same GPU that has always been in the Pi, now finally usable for ML workloads.

Why this is a bigger deal than a robot demo

The real story is what this does to the economics of edge AI. Cloud AI inference costs add up fast. Running a chatbot with 10,000 daily users means $200–500/month in API fees, forever. For a huge class of applications, Gemma 4 E2B and LiteRT-LM deliver production-quality AI at zero marginal cost. That changes what's economically viable.

There is also a privacy and reliability angle. LiteRT enables developers to build highly secure, self-contained systems, like intelligent robots and local AI agents, with zero cloud dependencies, ultra-low latency, and total data privacy. For industrial sensors, medical devices, or any deployment where data cannot leave the device, this matters more than benchmark scores.

One important caveat: Gemma 4 E2B and E4B are too heavy to be deployed on a Raspberry Pi 5 with Ollama, but work very well with Google AI Edge's LiteRT-LM. The memory efficiency gains are specific to the LiteRT runtime. Running the same model through a general-purpose tool like Ollama would require far more RAM than the Pi has available.

The model lineup for Raspberry Pi

Google offers a tiered set of models in the LiteRT community on Hugging Face, each targeting a different capability/memory tradeoff:

• Gemma 3 270M: Hyper-compact, designed for fine-tuned task-specific use cases like sentiment analysis or entity extraction.
• EmbeddingGemma 300M: Text embeddings for RAG, semantic search, and classification.
• Gemma 3 1B: Lightweight multilingual text model, good for summarization and content generation.
• Gemma 4 E2B: Tailored specifically for mobile and tight edge environments, featuring memory-mapped per-layer embeddings, ideal for continuous monitoring, fast text/image/audio inference, and edge-based speech processing where saving RAM is critical.
• Gemma 4 E4B: The sweet spot for performance and size, delivering noticeably stronger reasoning capabilities and frontier-level edge performance while remaining compact, and the perfect choice for complex multi-step planning without overwhelming the Pi's resources.

Getting started in two commands

LiteRT-LM runs the same model binary that powers Google's own Pixel Watch and Chromebook, which means your Raspberry Pi prototype and your production mobile deployment share the same inference stack. Setup is minimal:

pip install litert-cli
export HUGGING_FACE_HUB_TOKEN=<your_token>
litert lm run \
  --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \
  gemma-4-E2B-it.litertlm \
  --attachment=image.jpg \
  --prompt="Identify the main object in front of you and suggest an action."

The model download is approximately 1.5 GB. All models are available through the LiteRT Hugging Face Community, and the full Reachy demo source is in the LiteRT Samples GitHub repo. Google also announced that LiteRT integration and Gemma models are coming soon to Hailo AI accelerators, which would unlock dedicated NPU hardware acceleration through the same familiar workflow.

What needs updating in how we think about edge AI

The assumption that capable multimodal AI requires a GPU server or at least a dedicated NPU is now outdated for a meaningful slice of use cases. Gemma 4's E2B and E4B models bring multimodal AI (text, image, and audio) to phones and edge devices with 128K context windows and zero cloud dependency. The constraint used to be that you had to choose between capability and deployability. The PLE architecture, combined with a runtime purpose-built for edge constraints, removes that tradeoff for a large class of applications.

Combining Gemini API for complex cloud inference with Gemma 4 for local or on-device processing unlocks hybrid AI architectures that are private, low-latency, and cost-efficient, a combination that was difficult to achieve cleanly before this release. The robot demo is a proof of concept. The real target is every embedded system, smart camera, industrial controller, and offline-first application that previously had no viable path to running a capable language model locally.