Google Brings On-Device Voice to AI Edge Gallery With 100ms Startup

Google has added streaming speech recognition and expressive text-to-speech to AI Edge Gallery, giving developers a reference stack for voice applications that can run locally after the required models are installed. The release also introduces XR Language Teacher, a camera-based demo that identifies objects and text, answers spoken questions, and reads bilingual responses aloud.

Google’s announcement covers two related releases: speech APIs for application developers and a reference implementation that combines them with on-device models such as Gemini Nano and Gemma 4. Local inference can reduce network latency, keep audio and images on the device, and remove per-token hosted inference fees.

Camera in, bilingual speech out

XR Language Teacher turns a camera frame and spoken question into a bilingual lesson through a four-stage local pipeline:

1. The camera captures an object, sign, or menu while the microphone records the question.
2. Automatic speech recognition, or ASR, streams a transcript as the user speaks.
3. Gemma 4 receives the transcript and image, then produces a structured response that labels each language separately.
4. Text-to-speech, or TTS, routes those labeled phrases to the appropriate voices and accents while the interface displays a vocabulary card.

The structured response lets the application change voices within one sentence without asking the language model to generate audio directly. AI Edge Gallery uses the same speech tools in its main chat interface, where users can dictate prompts, play generated replies, enable automatic playback, and select highlighted sentences from a floating playback bar.

Choose the runtime by platform

Android developers gain smaller application packages because AICore manages the models at the operating-system level. Those files still occupy device storage, and applications may need to handle model availability or downloads before starting offline inference. Cross-platform developers manage their own LiteRT bundles and have more control over model selection and distribution.

Google’s latency and quality claims

Google reports the following Android results for devices with modern neural processing units, or NPUs. Word error rate measures transcript insertions, deletions, and substitutions against a reference transcript, with lower scores indicating fewer errors.

These figures are vendor benchmarks, so production teams should measure accuracy, memory use, thermal load, battery consumption, and sustained latency on their target devices. Microphone quality, background noise, language, speaking style, and available accelerators can materially change the results.

A modular pipeline cuts latency

Gemma 4 can process audio directly, but Google’s reference architecture assigns transcription and synthesis to smaller speech models. A dedicated ASR model can stream text into the language model, while a roughly 100-million-parameter TTS model begins synthesizing speech as response tokens arrive. Each component can also be updated or replaced independently.

Google’s compact TTS model uses few-step diffusion accelerated by an NPU. Diffusion synthesis refines an audio representation over several steps, while autoregressive systems generate output sequentially. Google says its implementation synthesizes a 27-second paragraph in about 500 ms and consumes less than half the active power of its traditional autoregressive comparison system.

LiteRT packages the components required by speech models into one .litertlm file. An ASR bundle may include encoder and decoder graphs plus a tokenizer; a TTS bundle may contain phonemizer dictionaries, voice embeddings, and a vocoder that converts generated features into audio. Consolidating those assets simplifies deployment and version management.

Start with runnable code

The Apache 2.0-licensed Gallery repository provides a working implementation that developers can adapt for several product categories:

• Offline museum and tour guides for locations with unreliable connectivity
• Hands-free cooking, maintenance, and repair assistants
• Live dictation and read-aloud accessibility tools
• Camera-based translation and language-learning applications
• Smart-glasses interfaces using the same camera, ASR, model, and TTS pipeline

Developers can install AI Edge Gallery from Google Play or the App Store, inspect the reference code, and download ready-to-run bundles from the LiteRT Community. The repository license covers the application code; individual open-weight models may have separate licenses and redistribution terms that should be checked before release.

Local execution avoids hosted inference charges and can operate without a network connection once every asset is present. Production planning still needs to account for model downloads, device storage, memory pressure, battery use, hardware variation, and any telemetry or cloud features added around the local pipeline.