NeMo Retriever NIM Family (Embedding & Reranking)

Accessed/verified: 2026-07-14. Docs: docs.nvidia.com/nim/nemo-retriever/ (text embedding and text reranking NIMs, latest).

What it is. The two model-serving legs of NeMo Retriever: an embedding NIM giving "easy access to state-of-the-art models that are foundational building blocks for enterprise semantic search applications" — text and image embeddings behind OpenAI-compatible APIs [embed-overview] — and a reranking NIM that "reorders citations by how well they match a query," which matters "especially when the retrieval pipeline involves citations from different datastores" [rerank-overview]. These are the exact endpoints the RAG blueprint, nv-ingest, and AI-Q pin.

Current model names

Embedding NIM support matrix [embed-matrix]:

The llama-nemotron-embed-* family supports float/int8/uint8/binary/ubinary output precisions and dynamic embeddings [embed-matrix] — the binary/quantized output is the vector-store cost lever (memory arithmetic in retrieval-rag-sota.md §3.3).

Reranking NIM support matrix [rerank-matrix]:

Numbers worth carrying [rerank-overview]: NVIDIA's own testing shows recall@5 rising from 0.5699 (dense search alone) to 0.7070 (dense + reranking); the cost side: "On an H100, reranking 500 passages will cost ~1,750ms." The developer marketing page additionally still lists the earlier llama-3.2-nv-embedqa-1b-v2 naming and reports ColEmbed models topping the ViDoRe V3 visual-retrieval leaderboard [nemo-retriever-page] — naming across generations is inconsistent; trust the support matrices for what a NIM container actually serves.

How you'd use it

Each NIM is one container from nvcr.io (NGC login with an API key), run with a GPU and called via OpenAI-compatible embeddings / ranking endpoints; MIG is not currently supported and the minimum host is an 8-core x86 with the NVIDIA Container Toolkit [embed-matrix]. In practice you rarely start them by hand — the RAG blueprint's deploy/compose/nims.yaml brings up the pair (nemotron-vlm-embedding-ms, nemotron-ranking-ms) alongside the LLM NIM [rag-selfhosted-deploy]. Prototyping against NVIDIA-hosted endpoints with an API key, then pulling the same containers on-prem, is the intended migration path [rag-hosted-deploy].

When to reach for it vs building yourself

Reach for the NIMs when you want retrieval models as versioned infrastructure: pinned containers, FP8/FP16-optimized engines per GPU SKU, an 8K-token embedder with binary output precisions, and a vision-language pair covering image-bearing corpora without a separate OCR-to-text detour. Self-hosting open models with your own vLLM/TEI stack wins when you need models outside the matrix (the MTEB/BEIR selection discussion in retrieval-rag-sota.md §7), CPU-only serving, or no NVIDIA-AI-Enterprise licensing dependency. The embedding-model-is-your-schema warning from the brief §2.3 applies doubly to a vendor family: migrating embedders means re-embedding the corpus, whoever serves the model.

Interview angle

"The current NeMo Retriever family is the llama-nemotron v2 line: embedding at 1b, a 300m distillation, and a vision-language embed-vl-1b-v2, all 2048-dim with 8K context on the text models and binary/int8 output options; reranking mirrors it at 1b, 500m, and rerank-vl-1b-v2. NVIDIA's own numbers make the reranker's case — recall@5 from 0.57 to 0.71 — but also its cost, about 1.75s for 500 passages on an H100, which is why rerank depth is a latency budget knob, wide-then-narrow."

Sources

embed-overview: {title: "NeMo Retriever Text Embedding NIM — Overview (docs.nvidia.com)", url: "https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html", accessed: "2026-07-14"}
embed-matrix: {title: "NeMo Retriever Text Embedding NIM — Support Matrix (model ids, dims, tokens, precisions)", url: "https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/support-matrix.html", accessed: "2026-07-14"}
rerank-overview: {title: "NeMo Retriever Text Reranking NIM — Overview (recall@5 0.5699→0.7070; H100 latency)", url: "https://docs.nvidia.com/nim/nemo-retriever/text-reranking/latest/overview.html", accessed: "2026-07-14"}
rerank-matrix: {title: "NeMo Retriever Text Reranking NIM — Support Matrix (model ids, token limits)", url: "https://docs.nvidia.com/nim/nemo-retriever/text-reranking/latest/support-matrix.html", accessed: "2026-07-14"}
nemo-retriever-page: {title: "NVIDIA NeMo Retriever — developer page (llama-3.2-nv-embedqa-1b-v2 naming; ColEmbed / ViDoRe V3 claim)", url: "https://developer.nvidia.com/nemo-retriever", accessed: "2026-07-14"}
rag-selfhosted-deploy: {title: "RAG Blueprint — self-hosted Docker deploy (nims.yaml; embedding/ranking container names)", url: "https://github.com/NVIDIA-AI-Blueprints/rag/blob/main/docs/deploy-docker-self-hosted.md", accessed: "2026-07-14"}
rag-hosted-deploy: {title: "RAG Blueprint — NVIDIA-hosted Docker deploy (API-key path)", url: "https://github.com/NVIDIA-AI-Blueprints/rag/blob/main/docs/deploy-docker-nvidia-hosted.md", accessed: "2026-07-14"}