# NVIDIA VSS Blueprint > NVIDIA's deployable reference stack for video AI agents that search, summarize, answer questions about and raise verified alerts on live or recorded video - Category: Multimodal Generation / Vision & Understanding - Language: Python - Licence: Apache-2.0 for the repository (Elasticsearch, Kibana and Redis config files dual-licensed SSPLv1 / AGPLv3; data under the NVIDIA Asset License); NIM models and NGC containers carry NVIDIA's own terms, and hosting NIM locally needs an NVIDIA AI Enterprise developer licence - Repository: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization - Website: https://build.nvidia.com/nvidia/video-search-and-summarization - Canonical page: https://ai-tldr.dev/tools/nvidia-vss/ - Updated: 2026-07-23 ## In short NVIDIA VSS Blueprint is NVIDIA's deployable reference stack for video AI agents that search, summarize, answer questions about and raise verified alerts on live or recorded video. It sits in the Vision & Understanding category of the Multimodal Generation tools and is open-source, written in Python and licensed Apache-2.0 for the repository (Elasticsearch, Kibana and Redis config files dual-licensed SSPLv1 / AGPLv3; data under the NVIDIA Asset License); NIM models and NGC containers carry NVIDIA's own terms, and hosting NIM locally needs an NVIDIA AI Enterprise developer licence. ## Overview The NVIDIA AI Blueprint for Video Search and Summarization (VSS) is a set of reference architectures for building vision agents and video analytics applications. It combines vision language models (VLMs), large language models, retrieval and NVIDIA NIM microservices so that people can search video streams or archives in natural language, summarize hours of footage, ask visual questions, generate reports, and detect and verify real-time alerts. The repository implements the blueprint and powers the hosted demo on build.nvidia.com. VSS is organised into three layers. Real-time video intelligence extracts features, embeddings and stream understanding from video and publishes the results to a message broker. Downstream analytics turns that metadata into trajectories, incidents and VLM-verified alerts. On top sits an agent that reaches video analytics data, incident records and vision tools through the Model Context Protocol (MCP), including VLM video understanding, embedding-based video search, long-video summarization and snapshot or clip retrieval. The pieces can be used inside existing applications, as standalone microservices, or as part of a larger agent. It is deployed with Docker Compose developer profiles (base, search, alerts and long-video summarization) or as a Brev Launchable, on NVIDIA hardware ranging from H100, RTX PRO 6000 Blackwell and L40S servers to DGX Spark and Jetson Thor edge devices, which run with a remote LLM. NVIDIA's prerequisites list an 18-core x86 CPU, 128 GB of RAM and a 1 TB SSD as the minimum, plus one to four GPUs depending on the profile. The repository code is Apache-2.0, but deployment pulls models and containers from NVIDIA's NGC registry with an API key, and hosting the NIM microservices locally requires an NVIDIA AI Enterprise developer licence. ## What it does - Reference agent workflows for Q&A and report generation, alert verification, real-time VLM alerts, natural-language video search and long-video summarization - Real-time video intelligence microservices that publish CV metadata, alerts and embeddings to a message bus for downstream use - Behavior analytics and alert verification that enrich detections into incidents and use a VLM to cut false positives - An MCP-based agent with tools for video understanding, semantic search, summarization, clip retrieval and Markdown or PDF reports - A web UI for uploading videos and chatting with the agent, with intermediate steps and a reasoning trace shown for each answer - Docker Compose profiles and Helm charts with shared, dedicated or remote LLM and VLM options, plus agentskills.io-compatible skills for deploying and operating VSS ## Getting started The quickstart deploys the base developer profile (video upload, Q&A and report generation) with Docker Compose on Ubuntu 24.04 or a supported NVIDIA edge OS. You need the NVIDIA Container Toolkit, Docker Engine 28.3.3 or later but below 29.5.0, Docker Compose v2.39.1+, the NGC CLI, an NGC API key and a Hugging Face token. If you lack the hardware, NVIDIA also documents a Brev Launchable deployment from a notebook in deploy/docker/scripts. 1. Clone the repository at a release tag Git LFS is required for the large files in the repo. ```bash git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git cd video-search-and-summarization git checkout tags/v3.2.1 git lfs install git lfs pull ``` 2. Set your credentials The deploy script reads your NGC and Hugging Face keys from the environment; --help lists the profiles and hardware options. ```bash export NGC_CLI_API_KEY='' export HF_TOKEN='' deploy/docker/scripts/dev-profile.sh --help ``` 3. Bring up the base profile for your GPU Pick the -H value that matches your hardware. On an H100 the LLM and VLM can share one GPU or get one each; set LLM_ENDPOINT_URL and add --use-remote-llm to point at a remote LLM instead (required on DGX Spark and Thor). ```bash # shared GPU deploy/docker/scripts/dev-profile.sh up -p base -H H100 # dedicated GPUs for LLM and VLM deploy/docker/scripts/dev-profile.sh up -p base -H H100 --llm-device-id 0 --vlm-device-id 1 # remote LLM export LLM_ENDPOINT_URL=https:// deploy/docker/scripts/dev-profile.sh up -p base -H H100 --use-remote-llm ``` 4. Check the agent is healthy, then open the UI Once the containers are up, the agent answers on port 8000 and the VSS UI is at http://:7777/. Drag a video into the chat and ask questions such as "Is the worker wearing PPE?" or "Can you generate a report for this video?". ```bash docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}" HOST_IP=$(ip route get 1.1.1.1 | awk '{print $7; exit}') curl -fsS "http://${HOST_IP}:8000/health" ``` 5. Tear it down ```bash deploy/docker/scripts/dev-profile.sh down ``` ## When to use it - Let operators ask plain-language questions of warehouse, smart-space or SOP-validation footage instead of scrubbing through hours of video - Cut false positives from computer-vision alerts by having a VLM verify each detection before anyone is paged - Summarize long recordings into a report by chunking the video and aggregating dense captions - Start a custom video analytics agent from NVIDIA's reference microservices rather than wiring detection, embeddings, storage and an LLM together from scratch ## Version history - 2026-07-23 v3.2.1: VSS 3.2.1 switched the default VLM for the real-time VLM microservice and the agent workflows to Cosmos Reason 3 Nano, moved the Smart City profile to 3.2.x microservices and updated CPU-based multimedia support. Older 2.x, 3.0 and 3.1 container images were deprecated.