Research Brief — Retrieval / RAG: SOTA Methodology (2025–2026)

Prepared: 2026-07-09. Audience: agentic-systems knowledge base; interview prep for multi-agent workflows, tool-calling, and stateful agentic frameworks. Sources are cited in-text by id; the full list (all URLs verified 2026-07-09) is in the Sources section in references.yaml format. Companion briefs: agent-loops-planning-sota.md, memory-sota.md, orchestration-algorithms-sota.md.

1. Executive summary

The RAG field did not get replaced by long context — it got absorbed into context engineering. The 2025 consensus (and 2026 practice) is that retrieval is one supplier of a model's limited "attention budget," alongside memory, tool outputs, and instructions [ragflow-2025-review] [context-rot-chroma]. Five shifts define current SOTA:

1. Retrieval became a tool inside the agent loop. Fixed retrieve-then-generate pipelines gave way to agentic RAG: the model decides when to search, what to query, which backend to use, and whether results suffice, looping until satisfied [agentic-rag-survey] [reasoning-rag-survey]. The frontier trains this behavior with RL (Search-R1 and descendants) rather than prompting it [search-r1-paper].
2. Hybrid + rerank is the standard quality stack. BM25 (or learned-sparse SPLADE) + dense embeddings fused with Reciprocal Rank Fusion, followed by a cross-encoder or late-interaction reranker, is the default production recipe; contextual chunk enrichment (Anthropic's Contextual Retrieval) cuts retrieval failures by up to 67% when combined with reranking [anthropic-contextual-retrieval] [rrf-paper] [splade-paper] [colbertv2-paper].
3. Quantization is now the core engineering lever in ANN indexes. Binary/rotational quantization with theoretical error bounds (RaBitQ, SIGMOD 2024) moved from research to default settings in Weaviate, Qdrant, Milvus and friends — 32× memory compression with rescoring recovers ~full recall [rabitq-paper] [weaviate-rq-blog] [qdrant-quantization].
4. Structure-aware RAG matured. GraphRAG (entity graphs + community summaries), RAPTOR (recursive summary trees), and HippoRAG (personalized PageRank over an open KG) address the two classic failures of flat vector RAG: global/sensemaking questions and multi-hop reasoning [graphrag-paper] [raptor-paper] [hipporag-paper].
5. Evaluation professionalized. BEIR/MTEB for retrievers and embeddings (MMTEB, ICLR 2025, is the current umbrella: 500+ tasks, 250+ languages), RAGAS/ARES for end-to-end RAG quality (faithfulness, answer relevance, context precision/recall), with LLM-as-judge calibrated against small human-labeled sets [beir-paper] [mmteb-paper] [ragas-paper] [ares-paper].

Bottom line for interviews: be able to (a) write the metric formulas, (b) explain HNSW/IVF-PQ mechanics and the quantization design space, (c) draw the hybrid+rerank pipeline, and (d) argue when agentic RAG / GraphRAG / long-context each win.

2. Retrieval and RAG evaluation

2.1 Classic IR metrics (know the formulas)

For a query set Q, ranked results, and binary/graded relevance labels (definitions per the standard IR literature; BEIR/MTEB use these throughout [beir-paper] [mteb-paper]):

• Recall@k — fraction of relevant documents that appear in the top k: Recall@k = |relevant ∩ top-k| / |relevant|. In RAG it is often reported as hit rate: the fraction of queries where at least one gold chunk is in the top k. This is the single most important retrieval metric for RAG, because the generator cannot recover what was never retrieved.
• MRR (Mean Reciprocal Rank) — MRR = (1/|Q|) Σ_q 1/rank_q, where rank_q is the position of the first relevant result. Sensitive only to the first hit; good for known-item search.
• nDCG@k — handles graded relevance and position discounting: DCG@k = Σ_{i=1..k} (2^{rel_i} − 1) / log₂(i + 1), nDCG@k = DCG@k / IDCG@k (IDCG = DCG of the ideal ordering). BEIR's headline metric is nDCG@10 [beir-paper].

2.2 Retriever/embedding benchmarks

• BEIR (2021): 18 heterogeneous zero-shot retrieval datasets. Key finding that still shapes practice: BM25 is a strong zero-shot baseline and dense retrievers trained on MS MARCO often underperform it out-of-domain — the original motivation for hybrid search [beir-paper].
• MTEB (2022): 8 task families (retrieval, reranking, clustering, classification, STS, …) across 58 datasets; the de-facto embedding leaderboard [mteb-paper] [mteb-leaderboard].
• MMTEB (ICLR 2025): community-driven expansion to 500+ tasks, 250+ languages, with cost-reduced evaluation via downsampling and hard-negative caching; introduces MTEB(eng, v2) — the split current leaderboards report [mmteb-paper].
• Watch for contamination: instruction-tuned embedding models increasingly train on MTEB-adjacent data; prefer held-out, domain-specific evals for model selection.

2.3 RAG-specific evaluation: RAGAS and ARES

RAGAS [ragas-paper] [ragas-faithfulness-docs] is reference-free (no gold answers required); an LLM judge decomposes and verifies. Core metric mechanics:

• Faithfulness — (1) decompose the generated answer into atomic claims; (2) for each claim, ask the judge whether it can be inferred from the retrieved context; (3) faithfulness = |supported claims| / |total claims| ∈ [0,1]. Example from the docs: answer "Einstein was born in Germany on 20 March 1879" against a context that says 14 March → 1 of 2 claims supported → 0.5 [ragas-faithfulness-docs].
• Answer relevancy — generate N synthetic questions from the answer (reverse engineering), embed them, and compute AR = (1/N) Σ_i cos(E(q_generated_i), E(q_original)). Low scores flag evasive/incomplete answers even when they are faithful.
• Context precision — are the retrieved chunks that matter ranked high? CP@K = Σ_k (precision@k · v_k) / |relevant chunks in top K| with v_k ∈ {0,1} a relevance indicator at rank k.
• Context recall — fraction of claims in a reference answer attributable to the retrieved context (this one needs a reference).

ARES [ares-paper] differs in two ways worth citing: it (1) fine-tunes lightweight LLM judges on synthetically generated in-domain (query, passage, answer) triples instead of prompting a frontier model, and (2) uses prediction-powered inference (PPI) with a small (~150+) human-labeled validation set to produce statistical confidence intervals around judge scores — addressing the calibration weakness of raw LLM-as-judge.

Practical 2025–26 guidance: use retrieval metrics (recall@k, nDCG) to debug the retriever, RAGAS-style triads for the pipeline, and end-task success (human-labeled or verified) as the gate; LLM judges must be spot-calibrated against humans (ARES formalizes this).

3. Vector index internals — with the quantization lens

3.1 HNSW (the default in-memory index)

Hierarchical Navigable Small World graphs [hnsw-paper] build a multi-layer proximity graph — a skip-list generalized to graphs:

• Insertion: each element draws a maximum layer l = ⌊−ln(unif(0,1)) · mL⌋ with mL = 1/ln(M) — an exponentially decaying layer distribution. The element is inserted into layers 0..l, connected to at most M neighbors per layer (2M on layer 0), with a diversity heuristic that prefers neighbors spread in different directions (critical on clustered data).
• Search: start at the entry point in the top layer; greedy descent layer by layer (each layer's search finds the local minimum); on layer 0 run best-first beam search with a candidate list of size efSearch. Recall/latency is tuned almost entirely by efSearch; build quality by M and efConstruction.
• Complexity: O(log N) search scaling; the payoff of the hierarchy is that long-range links in upper layers zoom in on the target region before fine search.
• Cost: everything (vectors + adjacency lists) lives in RAM — the reason quantized and disk-based variants exist.

3.2 IVF and Product Quantization (the FAISS lineage)

• IVF: k-means the corpus into nlist cells; at query time probe only the nprobe closest cells. Trades recall for scan cost; composable with any compression [faiss-paper].
• PQ (Product Quantization): split each D-dim vector into m subvectors; k-means each subspace into 2^b centroids (b usually 8); a vector is stored as m codes → m·b bits (e.g., 1536-dim float32 = 6144 B → 96 B at m=96, b=8, a 64× compression). Distance is computed by ADC (asymmetric distance computation): precompute per-query lookup tables T_j[c] = ‖q_j − centroid_{j,c}‖², then d̂²(q, x) = Σ_j T_j[code_j(x)] — m table lookups instead of D multiply-adds [faiss-paper].
• ScaNN / anisotropic vector quantization [scann-paper]: the key conceptual upgrade — quantization loss should be score-aware, not reconstruction-optimal. For MIPS, errors parallel to the datapoint direction distort top-scoring inner products more than orthogonal errors, so ScaNN minimizes h_∥‖r_∥(x)‖² + h_⊥‖r_⊥(x)‖² with h_∥ > h_⊥. This is the same philosophy as task-aware QAT: optimize the metric that matters downstream, not MSE.

3.3 Binary & rotational quantization — the current frontier (owner's home turf)

• Scalar quantization (int8): 4× compression, <1% recall loss, SIMD-accelerated distance kernels; the "free lunch" default [qdrant-quantization].
• Binary quantization: 1 bit/dim → 32× compression and up to ~40× speedup (Hamming/XOR kernels), but naive sign-quantization only works when dimensions are centered/decorrelated (empirically fine for OpenAI ada/embed-v3-class embeddings); production systems pair it with oversampling + rescoring — retrieve k·oversample candidates with binary codes, re-rank with original (or int8) vectors [qdrant-quantization].
• RaBitQ (SIGMOD 2024) [rabitq-paper]: the theoretical breakthrough. Apply a random orthogonal rotation P, quantize each rotated vector to its sign pattern (1 bit/dim), and use a corrected estimator of inner products that is unbiased with a probabilistic error bound shrinking as O(D^{-1/2}) — the first binary scheme with guarantees where PQ can fail arbitrarily. Extended to B-bit codes (SIGMOD 2025); directly productized: Weaviate's 8-bit Rotational Quantization (~4× compression, ~99% recall, faster distances via rotation
  • SIMD) is an explicit RaBitQ adaptation for HNSW [weaviate-rq-blog], and Qdrant/Milvus ship equivalents (Qdrant's docs also list asymmetric quantization and 1.5/2-bit variants) [qdrant-quantization].
• The QAT parallel worth making in an interview: random rotation before quantization is the same trick as Hadamard/rotation-based outlier smoothing in LLM weight/activation PTQ (QuaRot/SpinQuant style) — rotate to make the distribution isotropic so uniform quantizers stop being dominated by outlier coordinates. ANN search rediscovered QAT's core lessons: (1) rotate/whiten first, (2) optimize for the downstream score not reconstruction (anisotropic VQ ≈ loss-aware quantization), (3) keep a high-precision rescoring pass (≈ mixed-precision fallback).

3.4 Disk-based indexes (billion scale)

• DiskANN / Vamana (NeurIPS 2019) [diskann-msr]: build a flat graph (Vamana) with RNG-style pruning tuned for low hop count; store full vectors + adjacency on NVMe SSD, keep PQ-compressed codes in RAM to guide traversal; full-precision rescoring reads happen along the beam path. ~5–10ms latency, 95%+ recall@1 on a billion points on a single node — the blueprint for pgvectorscale's StreamingDiskANN and Milvus DiskANN.
• SPANN (NeurIPS 2021) [spann-paper]: inverted-index alternative — memory holds only centroids of posting lists; balanced posting lists with closure assignment (boundary points replicated into multiple lists) live on disk; query-adaptive pruning decides how many lists to read. Simpler I/O pattern than graph traversal; basis of several cloud-native designs.

4. Vector database landscape (2025–2026)

FAISS is a library (the design-space reference: flat/IVF/HNSW/PQ + refinement pipelines, GPU support) [faiss-paper]; the systems below are databases with filtering, replication, and hybrid search. Hybrid (sparse+dense) search and built-in quantization are now table stakes across all of them.

Selection heuristics that survive vendor churn: (1) below ~10–50M vectors, operational simplicity dominates — pgvector or an embedded option wins; (2) heavy metadata filtering demands filter-aware ANN (post-filtering on HNSW collapses recall); (3) memory budget → quantization menu matters more than raw QPS benchmarks; (4) hybrid search quality (real BM25

• fusion, not an afterthought) is the differentiator for RAG workloads.

5. The retrieval quality stack

5.1 Chunking

• Baselines: fixed-size token windows (200–800 tokens) with 10–20% overlap; recursive splitting on structural separators; document-structure-aware (headings, code AST, tables).
• Semantic chunking (split where embedding similarity between sentences drops) helps inconsistently; structure-aware splitting is usually the better investment.
• Late chunking [late-chunking-paper]: run the whole document through a long-context embedding model first, then mean-pool token embeddings per chunk boundary — each chunk's vector is conditioned on full-document context. Cheap, no LLM calls, needs a long-context embedder.
• Contextual Retrieval (Anthropic, Sep 2024) [anthropic-contextual-retrieval]: for each chunk, an LLM (Haiku-class) writes 50–100 tokens of document-situating context which is prepended before embedding and BM25 indexing. Numbers to remember: top-20 retrieval failure rate 5.7% → 3.7% (contextual embeddings, −35%), → 2.9% (+ contextual BM25, −49%), → 1.9% (+ reranking, −67%); ~$1.02 per million document tokens with prompt caching. This is the practical answer to "chunks lose context."

5.2 Hybrid search and fusion

• BM25: score(q,d) = Σ_t IDF(t) · tf/(tf + k1·(1 − b + b·|d|/avgdl)) — still the best zero-shot lexical matcher and the guard against dense embedding blind spots (IDs, code symbols, rare entities) [beir-paper].
• Learned sparse — SPLADE [splade-paper]: an MLM head expands each document/query into a sparse vocabulary-sized vector (log-saturated activations + FLOPS regularizer for sparsity); gets semantic expansion while keeping inverted-index serving.
• Fusion — RRF [rrf-paper]: RRF(d) = Σ_r 1/(k + rank_r(d)), k≈60. Rank-based, scale free, embarrassingly robust — the default way to merge BM25 + dense lists (and multi-query variants).

5.3 Reranking

Two architectures dominate the second stage:

• Cross-encoders: score each (query, doc) pair jointly with full attention — highest accuracy, O(candidates) LM passes, so rerank only top-50–200. Current production examples: Cohere Rerank, BGE-reranker, Qwen3-Reranker (0.6B–8B, trained jointly with the Qwen3 embedding family) [qwen3-embedding-paper].
• Late interaction — ColBERT [colbert-paper]: per-token embeddings with the MaxSim operator: S(q,d) = Σ_{i∈q} max_{j∈d} E_q_i · E_d_j. Two orders of magnitude cheaper than cross-encoders because document token embeddings are precomputed offline. ColBERTv2 adds residual centroid compression (~6–10× index shrink) + distillation-based training, and its PLAID engine prunes candidate generation with centroid interaction for large additional serving speedups [colbertv2-paper]. The late-interaction idea now extends to multimodal retrieval (ColPali-style page-image retrieval).

5.4 Query-side transformations

• Query rewriting (rewrite-retrieve-read): an LLM (or a small RL-trained rewriter) reformulates the user query for the retriever before retrieval [query-rewriting-paper].
• HyDE [hyde-paper]: generate a hypothetical answer document for the query, embed it, and search with that vector (optionally averaged with the query embedding) — zero-shot bridging of the query-document vocabulary gap. Weakness: hallucinated specifics can steer retrieval; less needed with strong instruction-tuned embedders.
• Multi-query / decomposition: issue several rewrites or sub-questions, RRF-fuse the result lists; sub-question decomposition is the standard entry point to multi-hop.

6. Advanced RAG patterns (and where the field moved)

6.1 Self-reflective and corrective pipelines

• Self-RAG [self-rag-paper]: train the generator with reflection tokens — Retrieve? (whether to fetch), ISREL (passage relevant?), ISSUP (output supported?), ISUSE (output useful?) — so the model itself decides when to retrieve and critiques its own grounding; decoding uses segment-level beam search weighted by critique scores. Elegant, but requires model training — which is why its ideas migrated into prompted agentic RAG rather than its implementation.
• CRAG (Corrective RAG) [crag-paper]: a lightweight retrieval evaluator (fine-tuned T5) scores retrieved docs → three-way action: Correct → decompose-then-recompose filtering of the docs; Incorrect → discard and fall back to web search with rewritten queries; Ambiguous → do both. Plug-and-play on any RAG stack; the canonical "retrieval can fail, plan for it" design.

6.2 Structure-aware RAG

• GraphRAG [graphrag-paper]: index time — LLM extracts an entity/relationship graph, Leiden community detection builds a hierarchy, LLM pre-writes community summaries; query time — global questions are answered map-reduce over community summaries, local questions traverse entities + their neighborhoods. Solves query-focused summarization ("what are the main themes?") where top-k chunk retrieval is structurally hopeless. Cost caveat: index construction is LLM-expensive; variants (LazyGraphRAG, LightRAG-style) defer summarization to query time.
• RAPTOR [raptor-paper]: recursively embed → cluster (GMM) → summarize chunks into a tree; retrieve across all abstraction levels ("collapsed tree"). Strong on questions whose evidence spans many chunks; simpler to build than a full KG.
• HippoRAG [hipporag-paper]: hippocampal-memory-inspired — open-IE triples build a schemaless KG; at query time, seed entities activate Personalized PageRank to do multi-hop evidence aggregation in a single retrieval step — comparable or better than IRCoT-style iteration while 10–30× cheaper and 6–13× faster, and up to 20% better than SOTA single-step retrieval on multi-hop QA.

6.3 Agentic RAG — retrieval as a tool (the 2025–26 mainstream)

The unifying move: stop hard-coding the pipeline; hand the retriever(s) to the agent as tools [agentic-rag-survey]. Taxonomy worth knowing [reasoning-rag-survey]:

• System-1 style (predefined workflows): routers that pick a retriever/strategy per query; adaptive RAG (classify query complexity → no retrieval / one-shot / iterative); CRAG/Self-RAG belong here conceptually.
• System-2 style (agentic search): the LLM interleaves reasoning with search calls — Self-Ask, IRCoT, Search-o1; multi-turn "deep research" agents are the production face.
• RL-trained search agents: Search-R1 [search-r1-paper] trains the policy (PPO/GRPO) to emit <search>query</search> actions inside its reasoning, with retrieved-token loss masking and outcome-only (exact-match) reward: +41% (Qwen2.5-7B) over RAG baselines. This line (R1-Searcher, DeepResearcher, etc.) is where "RAG" and "agent training" merged — see the companion brief.
• Engineering framing from Anthropic: prefer just-in-time retrieval (agent queries as needed) over pre-stuffed context; treat context as a budget; use sub-agents to explore and return compressed findings [anthropic-contextual-retrieval] (and the context-engineering post cited in the companion brief).

6.4 Long-context vs RAG — the debate's current state

• Lost in the Middle [lost-in-the-middle]: LLM accuracy on retrieved facts is U-shaped in position — strong at the start/end of context, weak in the middle; stuffing more context is not free.
• Context rot (Chroma, Jul 2025) [context-rot-chroma]: across 18 models (GPT-4.1, Claude 4, Gemini 2.5, Qwen3), reliability degrades non-uniformly with input length even on trivially simple tasks; semantic (non-lexical) matching and topically-similar distractors degrade fastest. Long context ≠ uniformly usable context.
• Self-Route [self-route-paper]: let the model judge whether retrieved chunks suffice; route to full long-context only when needed — comparable accuracy to always-long-context at a large cost reduction. LC vs RAG evaluation [lc-vs-rag-eval] adds the key nuance: strong closed models often win with full context, weaker/open models benefit from retrieval — the answer is model-capacity-dependent.
• Practical 2026 synthesis [ragflow-2025-review]: corpora don't fit in any window; cost and latency scale with tokens; so retrieval stays, but its job description changed — feed the right tokens into a budget-managed context (context engineering), with long context used for working sets, not for storage.

7. Embedding model selection

• Leaders (mid-2026, MTEB(eng, v2) / multilingual) [mteb-leaderboard]: Google gemini-embedding-001 tops the English board (~68.3 mean) [gemini-embedding-paper]; Qwen3-Embedding (0.6B/4B/8B, Apache-2.0) leads multilingual (~70.6 for 8B) and ships matched rerankers — the first time open weights lead the board [qwen3-embedding-paper]; Voyage, Jina v4 (multimodal), NV-Embed-class models fill out the top tier. Check the live leaderboard; churn is quarterly.
• Similarity metrics: for unit-normalized embeddings, cosine ≡ dot product ≡ monotone transform of L2 — pick per index support and normalize. Un-normalized dot product encodes popularity/magnitude priors (useful for recsys, risky for RAG).
• Matryoshka Representation Learning (MRL) [matryoshka-paper]: train with losses on nested prefixes (e.g., dims 64/128/256/…/full) so truncated embeddings remain valid — L = Σ_m c_m · L(f(x)[1:m]). This is what lets OpenAI text-embedding-3 / Gemini embeddings be truncated 3072→256 with minor recall loss; combine with int8/binary quantization for compound storage savings (dimension × precision — the two axes multiply).
• Instruction-tuned embeddings: current top models accept task instructions ("Represent this query for retrieving supporting documents…"); using the wrong prompt costs real recall — treat the instruction as part of the model contract [qwen3-embedding-paper].
• Fine-tuning: contrastive InfoNCE with in-batch negatives + mined hard negatives remains the recipe; LoRA on a strong open embedder with a few thousand domain pairs typically beats switching to a bigger general model; synthetic query generation (LLM writes queries for your chunks) bootstraps training data — the same synthetic-triple trick ARES uses for judges [ares-paper]. Fine-tune when domain vocabulary is specialized (chip-design RTL, EDA logs) and hybrid+rerank has plateaued.

8. What to add to the knowledge base

Proposed new pages (fits existing site structure: concept pages + references.yaml ids):

1. retrieval-evaluation.html — "Measuring Retrieval & RAG": recall@k/MRR/nDCG with formulas and a worked example; BEIR→MTEB→MMTEB lineage; RAGAS metric mechanics (faithfulness decomposition diagram); ARES and judge calibration (PPI); eval-driven RAG debugging flowchart (retriever vs generator failure attribution).
2. vector-indexes.html — "ANN Indexes & Quantization": HNSW mechanics (layer formula, efSearch); IVF-PQ + ADC; the quantization design space table (SQ/BQ/PQ/RaBitQ/anisotropic) with compression-vs-recall tradeoffs; DiskANN/SPANN; sidebar: "QAT lessons ANN rediscovered" (rotation, loss-aware objectives, mixed-precision rescoring) — a unique angle given the owner's onep-qat work.
3. rag-quality-stack.html — "The Retrieval Quality Stack": chunking (incl. late chunking + contextual retrieval with the 35/49/67% numbers); hybrid BM25+dense+RRF; rerankers (cross-encoder vs MaxSim late interaction); query rewriting/HyDE; a single end-to-end pipeline diagram.
4. advanced-rag.html — "Self-RAG, CRAG, GraphRAG, Agentic RAG": pattern cards with control-flow diagrams; long-context vs RAG debate box (lost-in-middle, context rot, self-route); bridge to the agents section ("retrieval as a tool"); Search-R1 as the training-time convergence point.
5. Update existing memory+RAG basics page to cross-link these and reframe under "context engineering."

9. The mathematical layer

The sections above state most results in prose; this section collects the underlying formulas, each attributed to its primary source, so the brief can be used to write the math down. Cross-references point to the section where each method is discussed (§2 metrics, §5 quality stack, §6 structure-aware RAG).

9.1 BM25, in full (§5.2)

The canonical form (Robertson & Zaragoza 2009, eq. 3.15) scores document d for query q by summing per-term weights over the query terms [bm25-foundations]:

score(q,d) = Σ_{t∈q} IDF(t) · tf_{t,d} / ( k1·((1−b) + b·(dl/avgdl)) + tf_{t,d} )

• IDF(t) = log[(N − n_t + 0.5) / (n_t + 0.5)] — the Robertson/Spärck Jones weight with no relevance information (N documents, n_t containing t), "a close approximation to classical idf" [bm25-foundations].
• k1 controls term-frequency saturation (each extra occurrence of t adds less); b interpolates document-length normalization — "setting b = 1 will perform full document-length normalisation, while b = 0 will switch normalisation off". The survey's guidance on ranges: "values such as 0.5 < b < 0.8 and 1.2 < k1 < 2 are reasonably good in many circumstances" [bm25-foundations].
• The common (k1+1) factor in the numerator (the Lucene-style variant) is a constant identical across terms and "does not affect the ranking produced" [bm25-foundations].

9.2 Training dense retrievers: DPR's objective and InfoNCE (§7)

DPR [dpr-paper] scores with a dot product of dual-encoder [CLS] vectors, sim(q,p) = E_Q(q)ᵀ · E_P(p) (BERT-base, d = 768), and trains by minimizing the negative log-likelihood of the positive passage against n negatives:

L(q, p⁺, p₁⁻…pₙ⁻) = −log [ e^{sim(q,p⁺)} / ( e^{sim(q,p⁺)} + Σ_{j=1..n} e^{sim(q,p_j⁻)} ) ]

In-batch negatives make this cheap: reuse the gold passages of the other questions in a mini-batch of size B as negatives, so one B×B similarity matrix trains every question against B−1 negatives at once (plus mined hard negatives, e.g. a top BM25 non-answer passage). The result that launched dense retrieval: +9–19% absolute over BM25 in top-20 retrieval accuracy [dpr-paper]. This is the retrieval instantiation of InfoNCE [infonce-paper]: L_N = −E[ log ( f(x⁺,c) / Σ_{x∈X} f(x,c) ) ] — "the categorical cross-entropy of classifying the positive sample correctly" — whose minimization maximizes a lower bound on mutual information, I(x;c) ≥ log(N) − L_N [infonce-paper]. The §7 fine-tuning recipe (in-batch negatives + mined hard negatives) is exactly this objective with a harder denominator.

9.3 Rank fusion: RRF and the canonical k = 60 (§5.2)

Given a set of rankings R (BM25, dense, multi-query rewrites…), each assigning rank r(d) to document d [rrf-paper]:

RRFscore(d ∈ D) = Σ_{r∈R} 1 / (k + r(d))

with k = 60 — "fixed during a pilot investigation and not altered during subsequent validation"; the pilot found k = 60 "near-optimal, but that the choice was not critical". The paper's rationale for the 1/(k+rank) shape: "while highly-ranked documents are more important, the importance of lower-ranked documents does not vanish" (as it would with an exponential decay), and the constant k "mitigates the impact of high rankings by outlier systems" [rrf-paper].

9.4 Late interaction: the MaxSim operator (§5.3)

ColBERT scores by summing, over query token embeddings, each token's best match among document token embeddings (eq. 3 of the paper) [colbert-paper]:

S(q,d) = Σ_{i∈[|E_q|]} max_{j∈[|E_d|]} E_{q_i} · E_{d_j}ᵀ

with similarity computed as "cosine similarity (implemented as dot-products due to the embedding normalization) or squared L2 distance". Because E_d is precomputed offline, query-time cost is |q|·|d| dot products rather than a Transformer pass per (q,d) pair — the asymmetry behind the two-orders-of-magnitude cheapness claim in §5.3; ColBERTv2's centroid-residual compression then attacks the resulting per-token storage cost [colbertv2-paper].

9.5 Personalized PageRank, as HippoRAG runs it (§6.2)

PageRank with a personalization (teleport) vector p⃗, in Haveliwala's formulation [topic-sensitive-pagerank]:

Rank⃗ = (1−α)·M·Rank⃗ + α·p⃗

Uniform p⃗ gives classic PageRank; "we can bias the computation to increase the effect of certain categories of pages by using a nonuniform … personalization vector for p⃗" — in random-walk terms, the walk restarts into the seed distribution with probability α. HippoRAG [hipporag-paper] seeds p⃗ with the query's extracted entities ("a personalized probability distribution defined over N, in which each query node has equal probability"), weighting each query node by node specificity s_i = |P_i|⁻¹ (P_i = the set of passages from which node i was extracted — an IDF analog), then ranks passages by the PPR mass of the nodes they contain. That single graph pass is the whole multi-hop machinery: "comparable or better performance than iterative retrieval like IRCoT while being 10-30 times cheaper and 6-13 times faster", and up to 20% better than state-of-the-art single-step retrieval on multi-hop QA [hipporag-paper].

9.6 RaBitQ's guarantee, precisely (§3.3)

For unit vectors o (data, quantized to ō) and q (query), RaBitQ's corrected estimator of the inner product is unbiased — E[ ⟨ō,q⟩ / ⟨ō,o⟩ ] = ⟨o,q⟩ — with the probabilistic bound (theorem 3.2):

| ⟨ō,q⟩/⟨ō,o⟩ − ⟨o,q⟩ | ≤ sqrt( (1−⟨ō,o⟩²)/⟨ō,o⟩² ) · ε₀/sqrt(D−1)

holding with probability at least 1 − 2·exp(−c₀·ε₀²) — i.e., O(1/√D) concentration, matching the asymptotic optimum of Alon & Klartag (2017) that the paper invokes to show its bound is asymptotically optimal. This is the precise content of §3.3's claim: "a sharp theoretical error bound" where PQ-style methods "do not have a theoretical error bound" [rabitq-paper]. Squared-distance estimates follow arithmetically, since ‖q−o‖² = 2 − 2·⟨o,q⟩ for unit vectors.

9.7 The metrics: lineage and a worked example (§2.1)

• nDCG's original definition (Järvelin & Kekäläinen 2002) uses linear gains — the graded relevance score itself — discounted by dividing "the document score by the log of its rank" (applied at ranks ≥ the log base), then normalized component-wise against the ideal ordering's vector [ndcg-paper]. The 2^rel − 1 exponential-gain form quoted in §2.1 is a later, widely used variant; for binary labels the two coincide (2¹−1 = 1, 2⁰−1 = 0), so they differ only under graded relevance. BEIR computes its headline "nDCG@10 for all datasets" with "the Python interface of the official TREC evaluation tool" [beir-paper].
• Worked example (one query; 3 relevant documents in the corpus; the system places relevant documents at ranks 2 and 5 of its top-5; binary labels, log₂(i+1) discount as in §2.1):
  • Recall@5 = 2/3 ≈ 0.67; hit rate = 1 (at least one relevant document in the top 5).
  • RR = 1/2 (first relevant at rank 2); MRR averages this quantity over the query set.
  • DCG@5 = 1/log₂(3) + 1/log₂(6) ≈ 0.631 + 0.387 = 1.018; IDCG@5 = 1/log₂(2) + 1/log₂(3) + 1/log₂(4) ≈ 2.131; nDCG@5 ≈ 1.018/2.131 ≈ 0.478.
  • Note what each metric "saw": recall ignored positions entirely, RR ignored the second hit, and only nDCG penalized the miss at rank 1 while crediting both hits.

9.8 Currency check (2026-07-09)

All formulas and figures in this section were verified against fetched primary sources on 2026-07-09 — the brief's preparation date — so there are no post-preparation developments to report; the main brief remains current as written.

Sources (references.yaml format; all URLs verified 2026-07-09)

ragas-paper: {title: "Ragas: Automated Evaluation of Retrieval Augmented Generation (Sep 2023)", url: "https://arxiv.org/abs/2309.15217", accessed: "2026-07-09"}
ragas-faithfulness-docs: {title: "Ragas Documentation — Faithfulness Metric", url: "https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/", accessed: "2026-07-09"}
ares-paper: {title: "ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems (NAACL 2024)", url: "https://arxiv.org/abs/2311.09476", accessed: "2026-07-09"}
beir-paper: {title: "BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models (NeurIPS 2021)", url: "https://arxiv.org/abs/2104.08663", accessed: "2026-07-09"}
mteb-paper: {title: "MTEB: Massive Text Embedding Benchmark (EACL 2023)", url: "https://arxiv.org/abs/2210.07316", accessed: "2026-07-09"}
mmteb-paper: {title: "MMTEB: Massive Multilingual Text Embedding Benchmark (ICLR 2025)", url: "https://arxiv.org/abs/2502.13595", accessed: "2026-07-09"}
mteb-leaderboard: {title: "MTEB Leaderboard — Hugging Face Space", url: "https://huggingface.co/spaces/mteb/leaderboard", accessed: "2026-07-09"}
hnsw-paper: {title: "Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs (2016, TPAMI 2018)", url: "https://arxiv.org/abs/1603.09320", accessed: "2026-07-09"}
faiss-paper: {title: "The Faiss Library (Jan 2024)", url: "https://arxiv.org/abs/2401.08281", accessed: "2026-07-09"}
scann-paper: {title: "Accelerating Large-Scale Inference with Anisotropic Vector Quantization (ScaNN, ICML 2020)", url: "https://arxiv.org/abs/1908.10396", accessed: "2026-07-09"}
rabitq-paper: {title: "RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for ANN Search (SIGMOD 2024)", url: "https://arxiv.org/abs/2405.12497", accessed: "2026-07-09"}
qdrant-quantization: {title: "Qdrant Documentation — Vector Quantization (scalar, binary, product)", url: "https://qdrant.tech/documentation/guides/quantization/", accessed: "2026-07-09"}
weaviate-rq-blog: {title: "8-bit Rotational Quantization: Improving the Speed-Quality Tradeoff of Vector Search — Weaviate", url: "https://weaviate.io/blog/8-bit-rotational-quantization", accessed: "2026-07-09"}
diskann-msr: {title: "DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node (NeurIPS 2019) — Microsoft Research", url: "https://www.microsoft.com/en-us/research/publication/diskann-fast-accurate-billion-point-nearest-neighbor-search-on-a-single-node/", accessed: "2026-07-09"}
spann-paper: {title: "SPANN: Highly-Efficient Billion-scale Approximate Nearest Neighbor Search (NeurIPS 2021)", url: "https://arxiv.org/abs/2111.08566", accessed: "2026-07-09"}
pgvector-github: {title: "pgvector — Open-Source Vector Similarity Search for Postgres", url: "https://github.com/pgvector/pgvector", accessed: "2026-07-09"}
late-chunking-paper: {title: "Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models (Jina AI, Sep 2024)", url: "https://arxiv.org/abs/2409.04701", accessed: "2026-07-09"}
anthropic-contextual-retrieval: {title: "Introducing Contextual Retrieval — Anthropic Engineering (Sep 2024)", url: "https://www.anthropic.com/engineering/contextual-retrieval", accessed: "2026-07-09"}
rrf-paper: {title: "Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods (SIGIR 2009)", url: "https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf", accessed: "2026-07-09"}
splade-paper: {title: "SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (SIGIR 2021)", url: "https://arxiv.org/abs/2107.05720", accessed: "2026-07-09"}
colbert-paper: {title: "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (SIGIR 2020)", url: "https://arxiv.org/abs/2004.12832", accessed: "2026-07-09"}
colbertv2-paper: {title: "ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (NAACL 2022)", url: "https://arxiv.org/abs/2112.01488", accessed: "2026-07-09"}
query-rewriting-paper: {title: "Query Rewriting for Retrieval-Augmented Large Language Models (EMNLP 2023)", url: "https://arxiv.org/abs/2305.14283", accessed: "2026-07-09"}
hyde-paper: {title: "Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE, Dec 2022)", url: "https://arxiv.org/abs/2212.10496", accessed: "2026-07-09"}
self-rag-paper: {title: "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024)", url: "https://arxiv.org/abs/2310.11511", accessed: "2026-07-09"}
crag-paper: {title: "Corrective Retrieval Augmented Generation (CRAG, Jan 2024)", url: "https://arxiv.org/abs/2401.15884", accessed: "2026-07-09"}
graphrag-paper: {title: "From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Microsoft, Apr 2024)", url: "https://arxiv.org/abs/2404.16130", accessed: "2026-07-09"}
raptor-paper: {title: "RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (ICLR 2024)", url: "https://arxiv.org/abs/2401.18059", accessed: "2026-07-09"}
hipporag-paper: {title: "HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (NeurIPS 2024)", url: "https://arxiv.org/abs/2405.14831", accessed: "2026-07-09"}
agentic-rag-survey: {title: "Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG (Jan 2025)", url: "https://arxiv.org/abs/2501.09136", accessed: "2026-07-09"}
reasoning-rag-survey: {title: "Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic RAG for Industry Challenges (Jun 2025)", url: "https://arxiv.org/abs/2506.10408", accessed: "2026-07-09"}
search-r1-paper: {title: "Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (Mar 2025)", url: "https://arxiv.org/abs/2503.09516", accessed: "2026-07-09"}
lost-in-the-middle: {title: "Lost in the Middle: How Language Models Use Long Contexts (TACL 2024)", url: "https://arxiv.org/abs/2307.03172", accessed: "2026-07-09"}
self-route-paper: {title: "Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach (Self-Route, EMNLP 2024)", url: "https://arxiv.org/abs/2407.16833", accessed: "2026-07-09"}
lc-vs-rag-eval: {title: "Long Context vs. RAG for LLMs: An Evaluation and Revisits (Jan 2025)", url: "https://arxiv.org/abs/2501.01880", accessed: "2026-07-09"}
context-rot-chroma: {title: "Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma Research (Jul 2025)", url: "https://www.trychroma.com/research/context-rot", accessed: "2026-07-09"}
ragflow-2025-review: {title: "From RAG to Context: A 2025 Year-End Review of RAG — RAGFlow (Dec 2025)", url: "https://ragflow.io/blog/rag-review-2025-from-rag-to-context", accessed: "2026-07-09"}
matryoshka-paper: {title: "Matryoshka Representation Learning (NeurIPS 2022)", url: "https://arxiv.org/abs/2205.13147", accessed: "2026-07-09"}
gemini-embedding-paper: {title: "Gemini Embedding: Generalizable Embeddings from Gemini (Mar 2025)", url: "https://arxiv.org/abs/2503.07891", accessed: "2026-07-09"}
qwen3-embedding-paper: {title: "Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (Jun 2025)", url: "https://arxiv.org/abs/2506.05176", accessed: "2026-07-09"}
bm25-foundations: {title: "The Probabilistic Relevance Framework: BM25 and Beyond (Robertson & Zaragoza, Foundations and Trends in IR, 2009)", url: "https://www.staff.city.ac.uk/~sbrp622/papers/foundations_bm25_review.pdf", accessed: "2026-07-09"}
dpr-paper: {title: "Dense Passage Retrieval for Open-Domain Question Answering (DPR, EMNLP 2020)", url: "https://arxiv.org/abs/2004.04906", accessed: "2026-07-09"}
infonce-paper: {title: "Representation Learning with Contrastive Predictive Coding (InfoNCE, Jul 2018)", url: "https://arxiv.org/abs/1807.03748", accessed: "2026-07-09"}
topic-sensitive-pagerank: {title: "Topic-Sensitive PageRank (Haveliwala, WWW 2002; university-hosted copy of the ACM paper)", url: "https://www.khoury.northeastern.edu/home/vip/teach/IRcourse/4_webgraph/notes/haveliwala02topicsensitive.pdf", accessed: "2026-07-09"}
ndcg-paper: {title: "Cumulated Gain-Based Evaluation of IR Techniques (Järvelin & Kekäläinen, ACM TOIS 2002; university-hosted copy)", url: "https://faculty.cc.gatech.edu/~zha/CS8803WST/dcg.pdf", accessed: "2026-07-09"}