NVIDIA Ships Open Blueprint That Searches Hours of Video in Real Time
NVIDIA open-sources a full reference stack that turns raw video streams and archives into searchable, summarizable, agent-queryable data using VLMs and MCP tools.
Takeaways
- NVIDIA released the open VSS Blueprint for GPU-accelerated video search, Q&A, and summarization agents.
- Architecture splits into real-time intelligence, downstream analytics, and agentic processing via MCP tools.
- Ships Cosmos-Reason2-8B VLM and Nemotron-Nano-9B-v2 LLM as NIM microservices.
- Includes workflows for alert verification, real-time anomaly detection, video search, and long summarization.
- Deployable via Docker Compose or Brev launchable notebook on x86, DGX-Spark, and Jetson THOR.
- Chunked GPU-parallel ingestion with vector plus graph database retrieval for long videos.
Searching hours of video footage has been one of those problems everyone talks about but few solve at production quality. NVIDIA is now shipping an open reference architecture that does exactly that, and the repo has been climbing GitHub trending.
The Video Search and Summarization Blueprint is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. It uses vision language models like NVIDIA Cosmos, LLMs like NVIDIA Nemotron, RAG, and NVIDIA NIMs. What ships here is the whole plumbing you would otherwise spend months wiring up yourself, not a single model or API.
Three layers doing different jobs
The blueprint splits video processing into three cleanly separated stages. Real-time video intelligence handles feature extraction, embeddings, and stream understanding, with results published to a message broker. Downstream analytics turns that metadata into trajectories, incidents, and verified alerts. Agentic and offline processing orchestrates tools for search, Q&A, summarization, and clip retrieval, including via the Model Context Protocol.
That MCP layer is where things get interesting. Instead of hard-coding a pipeline, the top-level agent treats video understanding, semantic search, summarization, and clip retrieval as tools it can call. If you already have an MCP-compatible agent stack, you can plug VSS into it without gluing together bespoke APIs.