Video Search & Summarization (VSS) Blueprint

Accessed/verified: 2026-07-14. Repo: github.com/NVIDIA-AI-Blueprints/video-search-and-summarization, current release VSS 3.2.0 (2026-06-16); at ~1.7k stars the most-starred repo in the blueprints org [vss-github] [blueprints-org].

What it is. "A suite of reference architectures for building GPU-accelerated vision agents and AI-powered video analytics applications" [vss-github] — it combines accelerated vision microservices, vision-language models, and LLMs so that video archives and live streams become something you can search, summarize, and question in natural language. It is retrieval-augmented generation with video as the corpus: the same ingest → embed → index → retrieve → generate shape as the RAG blueprint, with VLM captioning standing in for document parsing.

Architecture and models

The README names Cosmos-Reason2-8B and NVIDIA Nemotron-Nano-9B-v2 among its vision-language models [vss-github]. Processing is layered [vss-github]:

1. Real-time video intelligence — feature extraction, embeddings, and stream understanding on live video.
2. Downstream analytics — metadata enrichment, trajectories, incident detection, with metadata streams flowing through message brokers into analytics.
3. Agent / offline processing — tools for search, Q&A, and summarization over the indexed video, exposed via MCP so external agents can call them.

Ingestion chunks video, generates dense captions per chunk, and produces semantic embeddings for downstream retrieval [vss-github] — i.e. the "chunking" and "parsing" boxes of the enterprise-RAG diagram, applied to frames instead of pages.

How you'd use it

• Docker Compose is the primary deployment for bare metal or cloud instances; Helm charts live in /deploy for Kubernetes [vss-github].
• A Brev launchable gives the one-click rented-GPU path — the notebook deployment runs on 2x RTX PRO 6000 SE [vss-github] (launchables concept: overview page).
• Platforms: x86 Ubuntu 22.04/24.04, DGX-SPARK, and Jetson; GPU requirements vary by configuration [vss-github].
• What you get once it's up [vss-github]: video Q&A and report generation, natural language search across archives, long-form video summarization through clip analysis, and alert verification / real-time anomaly-detection workflows.

When to reach for it vs building yourself

Reach for it when the asset is video at scale — surveillance/operations archives, inspection footage, broadcast — and you'd otherwise be inventing a chunk-caption-embed pipeline, VLM serving, and a video-aware retrieval layer from scratch; the MCP tool surface also makes it a ready-made retrieval backend for an existing agent stack. Build yourself when you have modest clip counts (frame-sample + a hosted VLM + a vanilla vector store may suffice) or no NVIDIA GPU footprint — the blueprint's value is precisely its GPU-accelerated ingestion of long or live video, and it assumes that hardware.

Interview angle

"VSS is the retrieval stack generalized past documents: chunk the video, dense-caption each chunk with VLMs — the README names Cosmos-Reason2-8B and Nemotron-Nano-9B-v2 — embed the captions, then serve search, Q&A, and summarization as agent tools over MCP. Three layers: real-time stream understanding, downstream analytics like trajectories and incidents, and offline agentic tools. It's also a good example of NVIDIA's delivery pattern: compose for one node, Helm for scale, a Brev launchable for the one-click trial."

Sources

vss-github: {title: "Video Search and Summarization Blueprint — NVIDIA-AI-Blueprints/video-search-and-summarization README (3.2.0; models; layers; deploy; capabilities)", url: "https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization", accessed: "2026-07-14"}
blueprints-org: {title: "NVIDIA-AI-Blueprints GitHub organization — repo listing (VSS description, star count)", url: "https://github.com/orgs/NVIDIA-AI-Blueprints/repositories", accessed: "2026-07-14"}