DigUp searches inside screenshots, podcasts, and code

Apple’s Spotlight retrieves files by name and indexed text, leaving visual scenes, spoken passages, and concepts phrased differently much harder to find. DigUp is an open-source macOS menu bar app that searches the contents of selected folders and keeps extraction, indexing, and queries on the Mac.

DigUp wraps EmbeddingGemma 2, Google DeepMind’s multimodal embedding model, in a native Swift app. An embedding represents content as a list of numbers, placing related text, images, audio, and video near one another in a shared search space. Google released the Gemma 4-based model under Apache 2.0, according to its model announcement; DigUp uses the MIT license.

Describe it, open the source

Pressing Shift+Command+Space from any app opens DigUp’s search panel. A natural-language query returns the highest-ranked file and can jump directly to the relevant page, timestamp, or source-code line.

• zebra in a video opens the clip when the animal appears.
• where they talk about sleep jumps to the relevant podcast passage.
• the clause about pets in the lease opens the matching PDF page and highlights the text.
• code: retry a failed call with backoff finds an implementation by intent, even when the function has an opaque name.

Multilingual embeddings let an English query retrieve a note written in Bengali or Arabic, with support spanning more than 100 languages. DigUp also maintains a keyword index because semantic similarity can miss exact strings such as invoice numbers, symbols, and error codes.

From file bytes to search vectors

DigUp uses ImageIO, PDFKit, AVFoundation, and Vision to read images, documents, media, and screenshot text. Extracted content passes through an 8-bit quantized build of EmbeddingGemma 2, whose compressed weights reduce memory use. Inference runs through llama.cpp with Metal acceleration.

Vectors and keywords are stored in one SQLite file. Search ranks nearby vectors using cosine similarity, a measure of how closely their directions align, and then boosts exact keyword matches. Because every supported modality occupies the same 768-dimensional space, a text query can rank a video frame, audio segment, image, or PDF passage in one result set.

EmbeddingGemma 2 adds modality components in stages, allowing implementations to load only the components required for a given workload.

Encoding uses fixed token budgets: 280 tokens per image, 140 per video frame, and 25 per second of audio. One audio pass covers about 5.5 minutes, while text input is capped at 8,000 tokens. DigUp divides longer recordings into overlapping 30-second windows, samples one frame per video shot, and stores pointers back to the corresponding timestamps.

Local data, upfront compute

All extraction, embedding, indexing, and querying remain on the Mac. DigUp downloads an 865 MB model once, requires no account, and collects no telemetry. Its only routine network request is a daily update check, which users can disable.

Initial indexing time depends on the hardware and mix of files. The project’s examples range from a few minutes for several hundred files to about an hour for a large Downloads folder. A codebase the size of llama.cpp reportedly takes roughly 20 minutes to index and adds about 130 MB of storage. Subsequent filesystem changes generally become searchable within seconds.

The embedding helper runs as a separate process and exits after completing its work. During a search, the roughly 250 MB text encoder loads on demand, limiting persistent memory use while DigUp sits in the menu bar.

Blind spots and safeguards

• Platform: DigUp requires Apple Silicon and macOS 14 or later.
• OCR coverage: Apple’s Vision framework recognizes screenshot text in about 25 languages, creating a narrower language boundary than the embedding model’s 100-plus-language support.
• Unspaced scripts: Chinese, Japanese, and Thai screenshot text can lose exact-word matching because the keyword index depends on word boundaries. Semantic retrieval remains available.
• Uneven model quality: Google reports different performance levels across supported languages and no safety tuning. Applications built on the model need language-specific evaluation, access controls, and content policies.
• Repository access: DigUp reads source inside Git repositories only after code search is enabled. It skips likely keys, certificates, caches, and similar sensitive files by default.
• Cloud placeholders: Files stored only in iCloud remain unindexed until macOS downloads them locally.

One vector space, fewer encoders

DigUp implements the retrieval stage commonly used in retrieval-augmented generation systems. It selects relevant chunks and opens their sources; an assistant that drafts answers would require a separate language model and orchestration layer.

The shared embedding space covers text, images, audio, and video with one quantized model, while source code is embedded as text. This design consolidates model management, ranking, and hardware acceleration without requiring separate retrieval encoders for each media type.

Google reports active-memory figures of about 191 MB for quantized text-only weights and 567 MB for the full model on a Pixel 11 Pro. Actual Mac usage depends on loaded components and workload, but the mobile figures indicate the resource constraints the model targets.

The repository provides an inspectable Swift implementation of local content extraction, chunking, multimodal embedding, hybrid semantic and keyword ranking, deep links into source files, process isolation, and opt-in repository indexing.