HippoRAG

Full title: HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models Authors: Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su (The Ohio State University; Stanford University) Venue/Year: NeurIPS 2024 (arXiv v3, Jan 2025) arXiv: 2405.14831 Local PDF: /Users/moscalej/Documents/Onep/research-papers/2405.14831-hipporag.pdf

Why this paper matters

HippoRAG is the strongest existing argument that an agent's long-term memory should be a graph, not a pile of independent chunk embeddings. It transplants the hippocampal memory-indexing theory of human long-term memory into a RAG stack: an LLM plays the neocortex (turning passages into a schemaless knowledge graph via OpenIE), retrieval encoders play the parahippocampal regions (linking similar concepts), and Personalized PageRank over the graph plays the hippocampus performing pattern completion (§2.2). The payoff is single-step multi-hop retrieval: one PPR walk integrates evidence across passage boundaries, beating state-of-the-art retrievers by up to 20 points on multi-hop QA while being 10–30× cheaper and 6–13× faster online than iterative retrieval like IRCoT (§1, §4, Appendix G). For agentic systems it is the reference design for associative memory over a continuously growing corpus — new knowledge is integrated "by simply adding edges to its KG" rather than re-summarizing (§6.1).

The problem

RAG became the de facto long-term memory for LLMs, but "each new passage is encoded in isolation" (§1). That breaks any task needing knowledge integration across passages:

• Standard multi-hop QA is handled by iterative retrieve-generate loops (IRCoT), which are slow and expensive — multiple LLM calls per query.
• Path-finding multi-hop questions — the paper's sharper failure case (Figure 1, §5.3) — defeat even perfect iteration. "Which Stanford professor works on the neuroscience of Alzheimer's?" gives you two entry points and thousands of paths from each; no single passage mentions both traits, so similarity search cannot find Prof. Thomas Südhof unless the system already stores the association between disparate facts, the way human associative memory does.
• Offline-summarization alternatives (RAPTOR, GraphRAG) integrate at index time but "the summarization process must be repeated any time new data is added" (§6.1) — bad for a memory that must update continuously.

The mechanism

The design mirrors the hippocampal memory indexing theory (§2.1): the neocortex processes stimuli into higher-level features; the parahippocampal regions (PHR) route them; the C-shaped hippocampus stores an index of associations, achieving pattern separation (distinct experiences get distinct codes) and pattern completion (whole memories retrieved from partial cues, via the densely connected CA3 region). New information is integrated by changing only the index, never the cortical representations — exactly the update property you want in an agent memory.

Offline indexing — building the hippocampal index (§2.2–2.3)

An instruction-tuned LLM L processes each passage of corpus P with 1-shot prompts, in two steps: first extract named entities, then feed those entities into an OpenIE prompt that emits subject–relation–object triples (the two-step split balances "generality and bias towards named entities", §2.3). The union of noun-phrase nodes N and relation edges E is a schemaless KG — fine-grained, discrete features rather than one dense vector per passage, i.e., pattern separation. Then a retrieval encoder M (Contriever or ColBERTv2) adds synonymy edges E′ between any two nodes whose encoded representations have cosine similarity above threshold τ = 0.8 (§2.3, §3.4) — the PHR gluing near-duplicate mentions together. Indexing also records an |N|×|P| matrix P counting how many times each noun phrase appears in each passage (§2.3): the bridge from graph nodes back to retrievable text.

Online retrieval — pattern completion via PPR (§2.2–2.3)

1. The LLM extracts named entities from the query ("query named entities" C_q, e.g. Stanford, Alzheimer's).
2. Each c_i is linked to its most similar KG node by the retrieval encoder: r_i = e_k where k = argmax_j cosine_similarity(M(c_i), M(e_j)) — the resulting nodes R_q are the query nodes, the partial cues.
3. Personalized PageRank runs over the KG (|N| nodes, |E|+|E′| edges) with all restart probability concentrated on R_q, spreading probability only through the query nodes' joint neighborhood — the graph-search analogue of hippocampal pattern completion. Nodes reachable from both cues (Prof. Thomas) end up with high mass.
4. The output node distribution is multiplied by the P matrix to score and rank passages.

Node specificity — a "neurobiologically plausible" IDF (§2.3)

Global IDF would require aggregating corpus-wide counts at recall time, which the authors argue a brain (an "aggregator neuron" touching every node) could not do cheaply. Instead each node carries a purely local signal: s_i = |P_i|⁻¹, the reciprocal of the number of passages node i was extracted from. Query-node probabilities are multiplied by s_i before PPR, down-weighting promiscuous concepts and modulating each cue's whole neighborhood. In Figure 2 this is drawn as symbol size: Stanford outweighs Alzheimer's because it appears in fewer documents.

What the index actually looks like (Table 1)

Extracted KG statistics over the 1,000-question dev corpora give a feel for the scale and shape:

Note the ratio: nodes outnumber passages ~8×, and synonymy edges outnumber relational edges ~9×. The graph is wide and shallow — most of its connectivity is encoder-provided synonymy, with LLM triples supplying the semantic skeleton.

Configuration (§3.4, Appendix H)

• LLM L: GPT-3.5-turbo-1106 at temperature 0; retriever M: Contriever or ColBERTv2.
• Only two hyperparameters, tuned on 100 MuSiQue training examples: synonymy threshold τ = 0.8 and PPR damping factor 0.5 ("the probability that PPR will restart a random walk from the query nodes instead of continuing to explore the graph").
• PPR runs on python-igraph, on CPU (2× AMD EPYC 7513); performance "is rather robust to its hyperparameters" (§3.4).

The algorithm

# ---- Offline indexing ----
KG = empty graph;  P_matrix = zeros(|N| x |P|)
for each passage p in corpus P:
    ents  = LLM_NER(p)                          # 1-shot prompt
    triples = LLM_OpenIE(p, ents)               # 1-shot prompt, entities included
    add noun-phrase nodes and relation edges of triples to KG
    P_matrix[n, p] += count of node n in p
for each node pair (e_i, e_j):
    if cos(M(e_i), M(e_j)) > tau:               # tau = 0.8
        add synonymy edge (e_i, e_j)            # E'

# ---- Online retrieval (one query q) ----
C_q = LLM_NER(q)                                # query named entities
R_q = { argmax_e cos(M(c), M(e)) for c in C_q } # link cues to KG nodes
n_vec = zeros(|N|)
for r_i in R_q: n_vec[r_i] = s_i                # node specificity s_i = 1/|P_i|
n_vec = n_vec / sum(n_vec)                      # personalization distribution
n_prime = PPR(KG, personalization=n_vec, damping=0.5)
passage_scores = n_prime @ P_matrix             # aggregate node mass into passages
return top-k passages by passage_scores

Walkthrough:

• Lines 2–7 (pattern separation): every passage is decomposed into discrete triples; a passage about Prof. Thomas contributes nodes (Thomas, Alzheimer's) and (Stanford, employs, Thomas) even though no dense vector could represent both traits at once (Figure 2).

• Lines 8–10 (synonymy): the KG would otherwise fragment on surface forms; E′ edges connect "similar but not identical noun phrases" (§2.2). Table 1: on MuSiQue this adds ~146K–192K synonym edges to ~21.7K relational edges — the graph's connectivity mostly comes from the encoder.

• Lines 13–14 (cue linking): retrieval encoders only have to solve the easy problem — match a query mention to a node — not the hard one of matching a question to a multi-fact passage.

• Lines 15–18 (the PPR formulation): the personalized probability distribution n⃗ puts equal probability on each query node "and all other nodes have a probability of zero" (§2.3), scaled by node specificity. PPR then solves the standard fixed point

  n⃗′ = (1 − β) · W̃ᵀ n⃗′ + β · n⃗
  

  where W̃ is the normalized adjacency over triple + synonymy edges and β is the restart probability — the paper's "damping factor" of 0.5 (§3.4; the paper states the formulation in prose, not as an equation). Restarting only at query nodes biases mass toward their joint neighborhood — nodes like Professor Thomas that sit on paths from several cues — which is why one PPR run performs multi-hop reasoning "in a single retrieval step" (§1).

• Line 19 (back to text): node probabilities are converted to passage scores through the occurrence-count matrix P (§2.3), so retrieval still returns ordinary passages a reader LLM can consume.

Worked example (Figure 1/2, §5.3): for "Which Stanford professor works on the neuroscience of Alzheimer's?", indexing has already stored triples like (Thomas, researches, Alzheimer's) and (Stanford, employs, Thomas) from two different passages. Online, NER yields C_q = {Stanford, Alzheimer's}; both link to their KG nodes; PPR restarted at those two nodes concentrates probability on Professor Thomas, the node in their joint neighborhood; the P matrix maps that mass back to Thomas's passages. ColBERTv2 and IRCoT both return unrelated Stanford professors on this query (Table 7) because no single passage contains both cues.

Results that matter

How to read the numbers (§3.1–3.3):

• Benchmarks: 1,000 questions each from MuSiQue (answerable) and 2WikiMultiHopQA — the two genuine multi-hop tests — plus HotpotQA, included "for completeness" despite its known single-hop shortcuts; the retrieval corpus per dataset is all candidate passages (supporting + distractor) of the selected questions.
• Metrics: recall@2 / recall@5 (R@2, R@5) for retrieval; EM/F1 for QA; all-recall AR@k (Table 6) = fraction of queries where all supporting passages are in the top-k — the metric that isolates true multi-hop success.
• Baselines: BM25, Contriever, GTR, ColBERTv2, plus LLM-augmented indexing (Propositionizer, RAPTOR) and iterative retrieval (IRCoT) (§3.2).

Limitations & how to defend the findings

• "Isn't the graph doing nothing — maybe entity matching alone suffices?" Table 5's PPR ablations answer this directly: ranking passages by the query nodes alone drops average R@5 from 72.9 to 56.2, and naively adding their one-hop neighbors is worse (59.2). The multi-hop probability diffusion is the load-bearing part.
• "HotpotQA results are mediocre." True — ColBERTv2 beats HippoRAG single-step on HotpotQA R@2 (64.7 vs 60.5, Table 2). The authors' account: HotpotQA is a "much weaker test for multi-hop reasoning" with spurious single-hop cues (§3.1), and HippoRAG pays a concept-context tradeoff — its NER-centric cues discard contextual signal (Appendix F.2), partially fixable by ensembling (Appendix F/§4). Honest summary: HippoRAG wins where integration is genuinely required.
• "The pipeline is only as good as NER + OpenIE." The paper agrees: in a 100-error analysis on MuSiQue, 48% of errors are NER limitations (query cues too sparse), 28% incorrect/missing OpenIE, 24% PPR (Table 11, Appendix F). All components are used off-the-shelf with no fine-tuning (§7) — a floor, not a ceiling, but also a real fragility for deployment.
• "Indexing every passage through an LLM is expensive." Offline indexing is ~10× slower and ~$15 more per 10,000 passages than IRCoT's (Appendix G); mitigation shown in-paper: Llama-3.1-70B matches GPT-3.5 quality and indexes 10K docs in ~4 h on 4×H100 (Tables 5, 18). Online, the economics invert decisively (Table 17).
• "Does it scale beyond 10K-passage benchmark corpora?" Unproven, and the authors say so: "HippoRAG's scalability still calls for further validation" (§7). KG size, PPR cost, and OpenIE consistency on longer documents (Appendix F.4) are open.
• "Path-finding QA evidence is anecdotal." Correct — §5.3/Table 7 and the Appendix E case studies are qualitative; no path-finding benchmark existed. The quantitative core of the paper is path-following multi-hop QA.

Connections

• DPR (sibling page): HippoRAG is the structural complement to dense retrieval — it uses retrieval encoders for node linking and synonymy but replaces "query vector vs passage vector" scoring with graph diffusion, precisely because single-vector similarity can't represent multi-fact conjunctions.
• Graph memory in agents: the offline KG + incremental edge insertion is the same design argument behind agent memory graphs (see also Mem0's graph variant, Mem0g) — updates are local writes, not corpus re-summarization (§6.1 contrast with RAPTOR/GraphRAG).
• Multi-hop QA line: MuSiQue / 2WikiMultiHopQA / HotpotQA and the IRCoT iterative baseline situate this in the retrieve-and-reason literature; HippoRAG is complementary — plugging it into IRCoT adds up to ~20 points R@5 on 2Wiki (Table 3).
• PageRank lineage: Personalized PageRank as query-biased relevance dates to the original PageRank family; footnote 2 notes cognitive-science work correlating human word recall with PageRank output.
• LLM+KG synthesis: §6.3 places the work in the LLM↔KG "synergy" agenda — LLMs build the KG, the KG structures retrieval.
• Memory consolidation: the neocortex/hippocampus split (slow representations, fast index) parallels the extraction/consolidation split in Mem0 and the memory-sota brief's taxonomy of agent memory systems.

Reading guide

• Read first: §2.1 (the theory, half a page) → §2.3 (detailed methodology — the whole system in ~1 page) → Figure 2 (the one figure to study: both phases, all three brain-region analogies, node specificity drawn as symbol size).
• Then: Table 2 and §4 (main results), §5.1 + Table 5 (which components matter — the best table for skeptics), §5.2–5.3 + Table 7 (all-recall metric and the path-following vs path-finding distinction).
• Skimmable: §3 (setup), §6 (related work — read §6.1's RAG-as-memory paragraphs if you care about agent memory positioning).
• Appendices worth pulling: F (error taxonomy: 48/28/24 NER/OpenIE/PPR), G (cost tables 17–18 — the numbers behind the "10–30× cheaper" claim), A (worked example of the full pipeline).
• The one table to study: Table 5 — OpenIE alternatives, PPR alternatives, and both ablations in one place; it is the paper's causal story.