Google's EmbeddingGemma 2 Brings Multimodal Search to Your Device

Google has released EmbeddingGemma 2, a 740 million-parameter, open-weight embedding model designed for on-device search across text, images, audio, and video. The company also updated its AI Edge Gallery mobile app and introduced a Mac app called Google AI Edge Foresight to demonstrate local media search and meeting assistance.

An embedding model converts content into numerical vectors that capture semantic relationships. Applications can compare those vectors with cosine similarity or an approximate nearest-neighbor index, retrieving related items without sending each query to a cloud service. Inference stays on the device, which reduces network latency, limits per-query infrastructure costs, and keeps indexed content local.

One index for four media types

EmbeddingGemma 2 maps text, images, video frames, and audio into a shared vector space. A text query can therefore retrieve a photograph, an audio segment, or a moment within a video when their vectors are close enough.

Shared embeddings can simplify a retrieval pipeline that would otherwise require separate captioning, transcription, and text-embedding stages. Direct audio and image embeddings also preserve signals that captions or transcripts may omit, although applications still need to split media into useful chunks and attach timestamps or other metadata.

740M parameters with modular memory use

EmbeddingGemma 2 uses modular encoders, allowing applications to load only the modalities they need. Google reports the following active-memory figures on a Pixel 11 Pro:

Total application memory will be higher because the runtime, vector index, media buffers, metadata, and user interface also consume RAM. Applications that only search photographs can omit the audio encoder and reduce the model’s active footprint.

Google combines three techniques to reduce compute, memory, and index storage:

• Quantization-aware training: The model is trained to retain useful output at INT4 and INT8 precision, reducing weight size and making inference practical on consumer hardware.
• Matryoshka Representation Learning: The leading dimensions of each embedding remain useful as a smaller vector. Developers can truncate the default 768-dimensional output to between 128 and 512 dimensions, reducing index storage by up to eightfold.
• Adaptive input scaling: The runtime selects an appropriate computation graph for varying text lengths and vision-token budgets, avoiding unnecessary work on smaller inputs.

Google reports visual-embedding latency as low as 37.3 milliseconds on an M5 Pro GPU in a MacBook Pro, equivalent to about 26.9 images per second with a 70-token image budget. Its benchmark data also covers CPU, GPU, and NPU execution across mobile, desktop, and IoT reference devices. Actual latency will vary with input size, quantization, thermal limits, and concurrent workloads.

Foresight applies retrieval to live meetings

Foresight is an experimental Mac meeting assistant that captures system audio and microphone input, indexes transcripts and private files, and retrieves relevant material locally. System-level audio capture allows it to work across meeting platforms and continue processing without an active internet connection.

During a conversation, the app can expand shorthand notes with details from the live transcript. Its Live Assistant also detects questions and searches local documents, diagrams, and meeting content without requiring the user to type a query.

EmbeddingGemma 2 supplies the cross-modal retrieval layer by placing text, audio, and visual material in the same vector space. Any answer synthesis or note generation remains a separate application component, typically handled by another local model.

Phone demos search photos and video

The updated AI Edge Gallery includes Instant Media Search, which embeds local photographs and stores their vectors in SQLite. Search results are reranked as the user types, and multilingual queries can retrieve images without requiring translated filenames or manually assigned tags.

Google demonstrates the German query Katze schläft auf Tastatur, meaning cat sleeping on keyboard, retrieving the corresponding photograph in real time. The example shows how multilingual text embeddings can search visual content directly.

Video Moments Finder divides local videos into vision and audio chunks, embeds those segments, and links results to timestamps. A query such as dog catching a frisbee can open the matching moment directly. Direct audio embeddings allow relevant sounds or spoken content to contribute to retrieval without a separate transcription pass.

Three routes into an application

Google offers three integration levels, ranging from a managed Android API to direct runtime control:

All indexed items and incoming queries must use compatible model versions, dimensions, and preprocessing settings. Changing the model or embedding width generally requires rebuilding the index, while normalized vectors allow cosine similarity to be computed efficiently as a dot product.

Semantic routing without classifier training

Google also demonstrates EmbeddingGemma 2 as a zero-shot decision engine. An application embeds descriptions of available intents or actions, compares user input against those vectors, and selects the closest candidate without collecting a labeled training set or fine-tuning a classifier.

In Google’s chess-agent demo, the model evaluates 500 candidate moves in under 100 milliseconds. This approach can provide lightweight intent routing or action ranking for on-device agents. Applications should still enforce permissions, validate selected actions, and apply domain rules after semantic ranking.

What deployment still requires

• Indexing: Applications must chunk media, generate embeddings, store vectors and metadata, process updates, and remove deleted content.
• Retrieval tuning: Developers need to choose embedding dimensions, similarity thresholds, chunk sizes, candidate counts, and reranking strategies for their data.
• Resource planning: Published model-memory and latency figures exclude much of the surrounding application stack. Initial indexing can also consume significant time, energy, and storage.
• Generation: EmbeddingGemma 2 retrieves relevant material. Retrieval-augmented generation requires a separate language model to compose an answer from those results.
• Privacy controls: Local inference keeps embedding requests and indexed content on the device, while applications remain responsible for file permissions, model downloads, backups, telemetry, and data deletion.

For productivity, media, and agent applications, EmbeddingGemma 2 provides a compact retrieval layer that can search several content types through one index. Its practical advantage comes from combining multimodal search, modular memory use, and hardware-accelerated inference without adding a cloud request to every query.