--- title: "NVIDIA's Metropolis VSS 3 Lets Claude Summarize Hour-Long Video in 60 Seconds" subtitle: "VSS 3 adds 16 agent skills, Nemotron Nano Omni integration, and production-ready 3D multi-camera tracking so coding agents can query any video library in plain English" url: "https://alphasignal.ai/news/nvidia-s-metropolis-vss-3-lets-claude-summarize-hour-long-video-in-60-seconds" author: "AlphaSignal Newsroom" author_url: "https://alphasignal.ai/editorial-team" company: "NVIDIA AI" company_url: "https://alphasignal.ai/company/nvidiaai" published: "2026-06-24T19:00:01.000Z" updated: "2026-06-24T20:40:12.823Z" type: "News" topics: - "Development" - "Video" subtopics: - "Code Agents" - "Video Understanding" upvotes: 1361 read_time: "7 min" source_url: "https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/tree/main/skills?ncid=so-twit-426111&linkId=100000427854359" comments: 0 --- # NVIDIA's Metropolis VSS 3 Lets Claude Summarize Hour-Long Video in 60 Seconds > VSS 3 adds 16 agent skills, Nemotron Nano Omni integration, and production-ready 3D multi-camera tracking so coding agents can query any video library in plain English NVIDIA just shipped [Metropolis Blueprint for Video Search and Summarization (VSS) 3](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization), and the headline feature is deceptively simple: point a coding agent like Claude Code or Codex at a camera feed or a library of archived footage, type a natural language prompt, and the system does the rest. Under the hood, that simplicity is backed by a heavily rearchitected stack with 16 new agent skills, a unified open-source repo, and a new state-of-the-art 3D multi-camera tracking model. ## The problem that needed solving Enterprises, cities, and industrial operators are already drowning in video. Thousands of camera feeds run simultaneously, yet the moment something happens, the question is always the same: what actually occurred, and do I need to act? Traditional pipelines could store and record, but they couldn't reason. Getting insights out of recorded footage meant manual review or brittle rule-based systems that generated more false alarms than actionable alerts. VSS is organized into three areas of processing and analysis: real-time video intelligence (feature extraction, embeddings, and stream understanding with results published to a message broker), downstream analytics (enrichment of metadata into trajectories, incidents, and verified alerts), and agentic and offline processing (orchestrated tools for search, Q&A, summarization, and clip retrieval, including via the Model Context Protocol). ## 16 new agent skills: the real unlock The biggest shift in VSS 3 is how developers interact with the system. Instead of writing custom integration code, you drop a folder of skills into your coding agent and start prompting. Agent Skills are reusable, self-contained capabilities that follow the agentskills.io specification and package the prompts, reference data, and helper scripts a coding agent needs to operate a deployed VSS Blueprint. Each Skill maps a developer intent onto the corresponding VSS REST, VA-MCP, and VIOS calls. Skills are versioned alongside the blueprint in the VSS repository, exercised by a CI eval workflow on every change, and consumed by any compatible coding agent such as Claude Code or Codex at either deployment time or runtime. The agentskills.io format matters here: as of 2026, tools that support the SKILL.md format include Claude Code, Codex CLI, Gemini CLI, GitHub Copilot, Cursor, and several community tools like Cline, Windsurf, and OpenCode. The full set of VSS skills covers the entire operational surface of the blueprint: - `vss-search-archive` , open-vocabulary search across stored footage - `vss-summarize-video` , long-video summarization with timestamped highlights - `vss-manage-alerts` , query and verify upstream CV alerts - `vss-generate-video-report` , templated multi-video reports - `vss-deploy-detection-tracking-3d` , spin up 3D multi-camera tracking - `vss-deploy-detection-tracking-2d`, `vss-deploy-dense-captioning`, `vss-deploy-video-embedding` , individual microservice deployment - `vss-ask-video`, `vss-query-analytics`, `vss-setup-behavior-analytics`, `vss-manage-video-io-storage`, and more Because every Skill follows the agentskills.io specification, any compatible harness can load the same Skills folder without bespoke integration code. Validated harnesses include Claude Code, OpenAI Codex, OpenClaw, and NVIDIA's own NemoClaw sandbox. ## How the pipeline actually works VSS processes video in parallel chunks. Video files are processed as small chunks, typically a few seconds or minutes based on model and use case. The chunks represent a small portion of the video file, and processing of individual chunks is distributed across GPUs in parallel for better performance. A Vision Language Model (VLM) generates dense captions for each chunk, those captions are embedded and stored in a vector database, and an LLM then synthesizes them into summaries or search results. The retrieval pipeline, where Context-Aware RAG plays a major role, is responsible for processing the output of the ingestion pipeline and using it for various retrieval tasks like summarization of long video files, live streams, and Q&A on the indexed data. VLM captions, audio transcripts (if enabled), and associated metadata are processed, indexed, and stored in vector and graph databases. The result is dramatic: with the VSS blueprint, users can summarize a video 100x faster than watching in real time , an hourlong video can be summarized in text in less than one minute. ## Nemotron 3 Nano Omni: eyes and ears in one model VSS 3 integrates [Nemotron 3 Nano Omni](https://blogs.nvidia.com/blog/nemotron-3-nano-omni-multimodal-ai-agents/) for multi-video reports that span both video and audio. This is a new open omni-modal model , meaning it natively handles video, audio, images, and text in a single system rather than transcribing audio to text first and then reasoning over it. Nemotron 3 Nano Omni sets a new efficiency frontier for open multimodal models with leading accuracy and low cost, topping six leaderboards for complex document intelligence, and video and audio understanding. The key architectural trick is how it handles video: for video, Nemotron 3 Nano Omni uses a dedicated Conv3D tubelet embedding path. Instead of embedding each frame independently, every pair of consecutive frames is fused into a single "tubelet" before the vision encoder, halving the number of vision tokens the language model has to attend to. The model delivers 9x higher throughput than other open omni models with the same interactivity, resulting in lower cost and better scalability without sacrificing responsiveness. For VSS, this translates to faster, cheaper multi-stream report generation that captures audio context , conversations, alarms, ambient sound , alongside the visual scene. ## 3D multi-camera tracking hits production The other major addition is production-ready 3D multi-camera tracking powered by Sparse4D, which NVIDIA claims is the current state-of-the-art. This is a significant step up from 2D single-camera tracking. Where traditional systems lose track of an object the moment it moves between camera views, Sparse4D reasons across multiple camera angles simultaneously in 3D space , maintaining object identity even as it moves through a scene covered by overlapping cameras. The Spatial AI Data Utils library includes camera calibration and grouping, 3D-to-2D geometry, multi-cam 3D bounding box visualization, and detection and tracking evaluators covering HOTA, CLEAR, identity, and count metrics. This makes it practical for warehouse floor tracking, smart city intersections, and any environment where a single camera can't cover the full scene. ## Deployment and what you need Developers can deploy only the components they need, swap VLMs and LLMs , Cosmos Reason, Qwen3-VL, local NIMs, or remote model endpoints , without rebuilding services, and integrate the tools into any agent framework via the API Gateway / MCP layer. On hardware, the minimum validated configurations include: - 1x RTX Pro 6000, H100, H200, A100 (80 GB), B200, or DGX Spark for local deployment - AGX Thor and DGX Spark now supported with hybrid deployment (remote LLM) for edge scenarios - GH200 and GB200 platforms supported as of VSS 2.4.1 Because VSS runs on NVIDIA NIMs and locally hosted models, the full pipeline , from video ingest through VLM reasoning , can run inside a trusted, isolated network with no cloud dependency. The blueprint is free and open source on GitHub; running it locally requires an [NVIDIA AI Enterprise developer license](https://docs.nvidia.com/vss/latest/) to self-host NVIDIA NIM microservices. ## Who's already using it The real-world deployments give a sense of the scale this is designed for. Linker Vision's AI-powered application combines real-time video analytics with generative AI to understand complex urban events like floods or traffic accidents, delivering insights to 12 city departments and on track to scale from 30,000 city cameras to over 50,000, reducing incident response times by up to 80%. The National Hockey League used the VAST InsightEngine with the VSS blueprint to manage massive volumes of game footage, positioning itself to search through petabytes of video in sub-seconds for near-instant retrieval of highlights and in-game moments. On the manufacturing side, Pegatron's agents have reduced labor costs by 7% and defect rates by 67%. The practical use cases cluster around a few patterns: security and incident verification (replacing human review of flagged alerts), operations monitoring (warehouse safety, manufacturing QA), smart city infrastructure (traffic, crowd management), and media workflows (sports highlight retrieval, broadcast review). The new skills layer means all of these are now accessible to a coding agent with a plain-English prompt rather than a team of integration engineers. ## Sources - [X post by @nvidiaai](https://x.com/nvidiaai/status/2069858097930121319) - [NVIDIA VSS Blueprint GitHub Repository](https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization) - [VSS Official Documentation — Introduction](https://docs.nvidia.com/vss/latest/) - [VSS Release Notes](https://docs.nvidia.com/vss/latest/release-notes.html) - [NVIDIA Nemotron 3 Nano Omni Blog](https://blogs.nvidia.com/blog/nemotron-3-nano-omni-multimodal-ai-agents/) - [Transform Video Into Actionable Intelligence with AI Agents and Skills — NVIDIA Technical Blog](https://developer.nvidia.com/blog/transform-video-into-instantly-searchable-actionable-intelligence-with-ai-agents-and-skills/)