NVIDIA RAG Blueprint

Accessed/verified: 2026-07-14. Repo: github.com/NVIDIA-AI-Blueprints/rag, current release v2.6.0 (2026-06-04) [rag-github].

What it is. The org's "reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline" [blueprints-org]: one deployable artifact that wires the whole enterprise-RAG diagram — multimodal extraction, hybrid retrieval, reranking, generation with citations — out of NIM microservices, with Docker Compose and Helm deployments and OpenAI-compatible APIs on top [rag-github]. It is the concrete instantiation of the reference architecture in the enterprise-RAG brief §2/§4; this page is the operator's view.

Architecture and exact NIM names

Three services front the pipeline: a LangChain-based RAG orchestrator server (port 8081), the NeMo Retriever extraction ingestor server (port 8082), and a RAG UI [rag-github] [rag-hosted-deploy]. Ingestion pulls text, images, tables, charts, infographics, and audio out of documents; queries are embedded, matched "against enterprise data stored in a cuVS accelerated Vector Database," optionally reranked, and answered with citations [rag-github].

Default model wiring (README, v2.6.0) [rag-github]:

Optional NIMs [rag-github]: Llama 3.1 NemoGuard 8B (content safety & topic control — the NeMo Guardrails hook), Nemotron Nano Omni 30B A3B (reasoning/captioning), PaddleOCR, llama-nemotron-embed-vl-1b-v2 + llama-nemotron-rerank-vl-1b-v2 (multimodal retrieval), NVIDIA Riva ASR (audio).

Vector store choice: "The default is Elasticsearch. Another alternative is Milvus (GPU-accelerated)" [rag-github]. Elasticsearch buys the boring, battle-tested hybrid (dense + BM25 in one engine); Milvus buys cuVS/CAGRA GPU indexing when QPS or index-build time dominates — the CPU-HNSW-vs-GPU-CAGRA framing is in the brief §4.3.

Feature list (README) [rag-github]: hybrid dense+sparse search, reranking, multi-turn conversation, agentic RAG via a LangGraph plan-and-execute pipeline for multi-hop queries, optional reflection and guardrails, RAGAS evaluation, OpenAI-compatible APIs.

How you'd use it

Two Docker Compose paths, differing only in where the models run:

• NVIDIA-hosted models (light local footprint — you run only the servers, models are API calls using your NVIDIA/NGC API key) [rag-hosted-deploy]:
  1. export NGC_API_KEY="nvapi-..." then echo "${NGC_API_KEY}" | docker login nvcr.io -u '$oauthtoken' --password-stdin
  2. Clone the repo, source deploy/compose/.env
  3. docker compose -f deploy/compose/vectordb.yaml up -d
  4. docker compose -f deploy/compose/docker-compose-ingestor-server.yaml up -d
  5. docker compose -f deploy/compose/docker-compose-rag-server.yaml up -d
  6. Health: curl 'http://<host>:8082/v1/health?check_dependencies=true' (ingestor) and :8081/v1/health (RAG server); then open the UI, upload documents, query.
• Self-hosted on-prem models (the recommended single-node production shape; ~200GB free disk for model cache) [rag-selfhosted-deploy]: same sequence plus export MODEL_DIRECTORY=~/.cache/model-cache and, first, USERID=$(id -u) docker compose -f deploy/compose/nims.yaml up -d to bring up the NIMs locally. Running containers you should see: nim-llm-ms, nemotron-ranking-ms, nemotron-vlm-embedding-ms, elasticsearch, seaweedfs, rag-server, rag-frontend, ingestor-server [rag-selfhosted-deploy].

Kubernetes: Helm charts, plus Red Hat OpenShift support behind an openshift.enabled flag [rag-github]. Constraints worth quoting: Docker Engine >= 24.0 ("Docker Engine 29.5.x is not supported for this release"), Docker Compose >= 2.29.1 [rag-hosted-deploy]. The docs tree also ships a retrieval-only deployment, a Python client library mode, and a containerless "lite mode" notebook [rag-docs-tree]. (A hosted trial exists on build.nvidia.com, but that page would not load over plain HTTP this session — the NVIDIA-hosted compose path above is the verified equivalent.)

When to reach for it vs building yourself

Reach for it when you want the pinned, co-tested NVIDIA stack — extraction NIMs + embed/rerank + Nemotron generation on your own GPUs, with data-sovereignty and an OpenAI-compatible seam — and your pipeline shape matches its opinions (its extraction, its two vector stores). Build yourself (or unbundle it) when you need a different embedder/store/LLM mix, CPU-only economics, or a thin pipeline where a few hundred lines of LangChain/LlamaIndex beat operating ~8 containers; the component menu per stage is in the brief §2. Middle path: keep nv-ingest and the retriever NIMs, swap the generator — everything speaks OpenAI-compatible APIs.

Interview angle

"The RAG blueprint is NVIDIA's whole-diagram artifact: v2.6.0 pins Nemotron 3 Super (nemotron-3-super-120b-a12b) for generation, llama-nemotron-embed-1b-v2 and llama-nemotron-rerank-1b-v2 for retrieval, and the Nemotron page-elements / table-structure / graphic-elements / OCR NIMs for extraction. Elasticsearch is the default store with GPU-accelerated Milvus (cuVS/CAGRA) as the alternative — that's the CPU-HNSW-vs-GPU-graph-index trade. I'd deploy compose-on-one-node first, hosted NIMs to prototype, self-hosted for data sovereignty, Helm when it has to scale."

Sources

rag-github: {title: "NVIDIA RAG Blueprint — NVIDIA-AI-Blueprints/rag README (v2.6.0; default model ids; features; Elasticsearch/Milvus)", url: "https://github.com/NVIDIA-AI-Blueprints/rag", accessed: "2026-07-14"}
rag-hosted-deploy: {title: "RAG Blueprint — Deploy with Docker (NVIDIA-hosted models): commands, ports, prerequisites", url: "https://github.com/NVIDIA-AI-Blueprints/rag/blob/main/docs/deploy-docker-nvidia-hosted.md", accessed: "2026-07-14"}
rag-selfhosted-deploy: {title: "RAG Blueprint — Deploy with Docker (self-hosted models): nims.yaml, 200GB cache, container list", url: "https://github.com/NVIDIA-AI-Blueprints/rag/blob/main/docs/deploy-docker-self-hosted.md", accessed: "2026-07-14"}
rag-docs-tree: {title: "RAG Blueprint — docs/ directory (deployment guide inventory: helm, MIG, retrieval-only, python client, lite mode)", url: "https://github.com/NVIDIA-AI-Blueprints/rag/tree/main/docs", accessed: "2026-07-14"}
blueprints-org: {title: "NVIDIA-AI-Blueprints GitHub organization — repo listing (rag description)", url: "https://github.com/orgs/NVIDIA-AI-Blueprints/repositories", accessed: "2026-07-14"}