Generative Agents

Full title: Generative Agents: Interactive Simulacra of Human Behavior Authors: Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein (Stanford; Google Research; Google DeepMind) Venue/year: UIST '23 (ACM Symposium on User Interface Software and Technology), 2023 — arXiv:2304.03442 (v2: Aug 2023) Links: arXiv · local PDF: research-papers/2304.03442-generative-agents.pdf

Why this paper matters

This is the "Smallville" paper: 25 LLM-driven agents living in a Sims-like town, waking up, cooking breakfast, gossiping, and — from a single seed intent — autonomously organizing a Valentine's Day party that five agents actually attend (§7.1). Its lasting contribution is not the demo but the memory architecture that made the demo possible: the memory stream (a timestamped, append-only record of experience), a retrieval function scoring memories by recency, importance, and relevance, reflection that synthesizes observations into higher-level inferences, and plan-then-react behavior — all fed back into the same stream (§4, Figure 5). Published mid-2023, it predates MemGPT and is the single most-copied memory design in agent systems: nearly every "agent memory" library since implements some variant of its three-term retrieval score, and its reflection mechanism is the ancestor of today's memory-consolidation pipelines. It is also an HCI paper: the evaluation currency is believability, measured by human rankings, not task accuracy.

The problem

Believable simulated humans had been a goal for four decades (The Sims, NPC behavior trees, cognitive architectures like SOAR-based NPCs — §2.2), but every prior approach either hand-authored behavior (finite-state machines, behavior trees — brittle, can't handle open-world interaction) or used RL (works only where reward is definable, not for open-ended social life). LLMs can simulate human behavior at a single time point, but a raw LLM has no persistence: "the resulting agents may not react based on the agent's past experiences, may not make important inferences, and may not maintain long-term coherence" (§4). The core obstacle is stated crisply in §4.1: the full record of an agent's experience is far larger than the context window, and naive summarization produces uninformative mush — ask Isabella "What are you passionate about?" against a summary and you get generic cafe talk; retrieve the right memories and you get the Valentine's Day party. So the problem is: which handful of memories, out of a life's worth, should condition this moment's behavior — and how do raw observations ever become self-knowledge?

The mechanism

The memory stream (§4.1)

A single database of memory objects, each holding a natural-language description, a creation timestamp, and a last-access timestamp. The basic unit is an observation — an event the agent directly perceives ("Isabella Rodriguez is setting out the pastries", "The refrigerator is empty"). Reflections and plans are also memory objects in the same stream, so retrieval treats experience, insight, and intention uniformly.

Retrieval scoring (§4.1)

Given the agent's current situation as query, every memory is scored:

score = α_recency · recency + α_importance · importance + α_relevance · relevance

with all α = 1 in the implementation and each component min–max normalized to [0, 1]. Term by term:

• Recency — exponential decay over sandbox game hours since the memory was last retrieved, decay factor 0.995. Note the subtlety: decay runs from last access, not creation — retrieving a memory refreshes it, an attentional rehearsal effect.
• Importance — an absolute, query-independent "poignancy" score assigned once, at creation, by asking the LLM to rate the memory 1–10 ("1 is purely mundane (e.g., brushing teeth, making bed), 10 is extremely poignant (e.g., a break up, college acceptance)"). The paper reports 2 for "cleaning up the room" and 8 for "asking your crush out on a date". This is the cheap trick that filters a life of "desk is idle" noise.
• Relevance — cosine similarity between the memory's embedding and the query's embedding. Query-dependent; this is standard dense retrieval.

Top-ranked memories that fit in the context window are put in the prompt (Figure 6 shows the scored ranking for "What are you looking forward to?").

Reflection (§4.2)

With only raw observations, agents can't generalize — asked who to spend time with, Klaus picks whoever he saw most often rather than the fellow researcher he shares a passion with. Reflections fix this. They are triggered when the sum of importance scores of recent events exceeds a threshold (150) — roughly two or three times a day in practice. The process: (1) feed the LLM the 100 most recent memories and ask for the "3 most salient high-level questions we can answer about the subjects"; (2) use those questions as retrieval queries; (3) prompt for 5 insights with citations to the evidence records ("insight (because of 1, 5, 3)"); (4) store each insight as a reflection in the stream, with pointers to its sources. Since reflections can cite reflections, this builds a reflection tree: observations at the leaves, increasingly abstract self-knowledge toward the root ("Klaus Mueller is highly dedicated to research", Figure 7).

Planning and reacting (§4.3)

Prompt-only agents are myopic — Klaus will eat lunch at 12:00, then again at 12:30 ("Optimizing for believability in the moment sacrifices believability over time"). Plans are memory objects with location, start time, and duration, generated top-down recursively: first a day agenda in 5–8 broad chunks (from the agent's summary description plus a digest of the previous day), then decomposition into hour-long chunks, then 5–15 minute chunks. On every time step the agent perceives; the LLM is asked — given the observation plus a retrieved context summary built from two queries ("What is [observer]'s relationship with [observed entity]?", "[Observed entity] is [action status]") — whether to continue the plan or react; a reaction regenerates the plan from that point (§4.3.1). Dialogue is generated the same way, each utterance conditioned on the speaker's memories of the other agent plus the dialogue history (§4.3.2). Implementation: gpt-3.5-turbo (GPT-4 API was invitation-only at the time, §4).

The algorithm

The core loop reconstructed from §4 and Figure 5, with the three sub-procedures:

# --- retrieval (§4.1) ---
def retrieve(query, stream):
    for m in stream:
        rec = 0.995 ** hours_since_last_access(m)         # game-time decay
        imp = m.importance / 10                            # LLM 1-10, set at creation
        rel = cos(embed(m.text), embed(query))
        m.score = norm(rec) + norm(imp) + norm(rel)        # α's all = 1, min-max norm
    return top_ranked_that_fit_context(stream)

# --- reflection (§4.2) ---
def maybe_reflect(stream):
    if sum(m.importance for m in recent_events) > 150:     # ~2-3x per game day
        questions = LLM("3 most salient questions", last_100(stream))
        for q in questions:
            evidence  = retrieve(q, stream)
            insights  = LLM("5 insights, cite evidence ids", evidence)
            stream.append(Reflection(insights, pointers=cited_ids))   # tree grows

# --- planning (§4.3) ---
def plan_day(agent):
    outline = LLM(agent.summary + yesterday_digest)        # 5-8 broad chunks
    return recursively_decompose(outline)                  # -> hours -> 5-15 min
stream.append(plan)                                        # plans are memories too

# --- the per-timestep loop (§4.3.1) ---
loop each sandbox time step:
    obs = perceive()                                       # within visual range
    stream.append(obs, importance=LLM_poignancy(obs))
    ctx = summarize(retrieve("relationship with X", stream),
                    retrieve("X is <status>", stream))
    if LLM("should agent react?", agent.summary, obs, ctx):
        plan = regenerate_plan_from_now()
        if reaction_is_interaction: run_dialogue()         # memory-conditioned turns
    act(next_step_of(plan))                                # emitted as NL, then emoji
    maybe_reflect(stream)

Walkthrough:

• retrieve — the three terms answer three different questions: is it fresh in mind? (recency), did it ever matter? (importance), does it bear on now? (relevance). Summing normalized scores rather than multiplying means a memory can win on one strong dimension: a wedding from months ago (importance) can beat this morning's idle desk (recency). Access-time refresh makes retrieval self-reinforcing — remembered things stay rememberable.
• importance at creation — scored once and cached; cheap, but it means importance never gets revised in light of later events (the paper lists fine-tuning the score functions as future work, §8.2).
• maybe_reflect — the importance-sum trigger is elegant: reflection happens after eventful periods, not on a timer. Citation pointers make each insight auditable back to raw experience, and letting reflections cite reflections is what produces genuine abstraction rather than one-hop summaries.
• plan_day / recursive decomposition — coarse-to-fine planning keeps the day globally coherent while leaving room to react; regenerating the plan from the reaction point onward preserves the completed past.
• act — actions are natural-language statements translated into sandbox movement and object state changes by separate LLM queries (§5); the architecture never touches game code directly.

Results that matter

Evaluation design worth remembering: believability was measured by 100 Prolific evaluators ranking interview responses (25 questions across self-knowledge, memory, plans, reactions, reflections) from five conditions, converted to interval ratings via TrueSkill (§6.1–6.4). All pairwise differences significant (Dunn post-hoc, p < 0.001) except crowdworker vs fully-ablated (§6.5.1).

Limitations & how to defend the findings

• "Believability rankings are subjective — where's the task metric?" By design: believability is the paper's HCI dependent variable with lineage back to believable-agents research (§6.1). The defense is the ablation structure, not the absolute scores — each removed component (reflection, planning, observation) monotonically drops the rating (§6.5.1), which is a causal claim about the architecture, robust to what "believable" means to any individual rater.
• "Agents beat humans? Surely that's overselling." The paper is careful: the crowdworkers watched a replay and roleplayed the agent; they are "a helpful comparison point," explicitly "not the maximal human expert performance" (§6.2, §8.2). The right reading is "the architecture clears a competent-human-roleplay bar," not "superhuman believability."
• "The agents hallucinate and embellish." Documented candidly in §6.5.2: retrieval failures (Rajiv missing Sam's candidacy he had heard), incomplete fragments (Tom certain about discussing the election at a party he isn't sure exists), embellishments (Isabella inventing Sam's "announcement tomorrow"), and world-knowledge leakage (Yuriko's neighbor "Adam Smith" who "authored Wealth of Nations"). End-to-end, hallucination measured 1.3% on awareness questions (§7.1.2) — low but nonzero, and only on one question type.
• "Two game days, 25 agents — does it scale or persist?" Conceded (§8.2): the eval window is short, the simulation cost "thousands of dollars in token credits" and took "multiple days" of wall-clock time. §7.2 adds an honest scaling symptom: as memory grows, agents pick less-typical locations and norms erode (agents lunching at the bar; entering a one-person dorm bathroom together; shopping after closing). Growing memory stresses retrieval and action-space selection, not just storage.
• "Behavior is contaminated by the base model's instruction tuning." The paper noticed it first (§7.2): agents are overly formal and overly cooperative (Mei's stilted greetings to her own husband; Isabella agreeing to nearly every party suggestion). A real confound for "emergence" claims — some sociability is RLHF, not architecture.
• "Is it safe to build?" §8.3 flags parasocial attachment, error cascades, deepfake/persuasion misuse (proposing audit logs of inputs/outputs), and over-reliance replacing real human input in design processes.

Connections

• The retrieval score is CoALA's §4.3 canonical example of a retrieval action ("recency (rule-based), importance (reasoning-based), and relevance (embedding-based)"), and the whole system fills CoALA's richest Table-2 row: all four action types, episodic and semantic memory.
• Reflection is memory consolidation — episodic-to-semantic distillation with provenance pointers. It is the design ancestor of Reflexion's verbal self-feedback, Letta/Mem0-style background consolidation jobs, and AriGraph's semantic-graph learning; the importance-sum trigger anticipates event-driven (rather than scheduled) consolidation.
• Generative Agents vs MemGPT: complementary halves of the memory problem. This paper never edits or evicts — the stream only grows, and §7.2 shows the strain; MemGPT is precisely the missing eviction/self-editing layer. Conversely, MemGPT's flat recall search has no importance or recency term — production systems (e.g. Letta's retrieval, Zep's rerankers) blend both papers.
• The importance score lives on as the LLM-scored salience filter in modern memory pipelines (Mem0's fact extraction, "memory-worthiness" gates) — a one-token LLM call as noise filter remains the best cost/benefit trick in the stack.
• Smallville → multi-agent social simulation: information-diffusion and network-density measurements (§7.1) established the evaluation template later used for LLM social-simulation work, and the environment-as-tree grounding (§5.1) prefigures world-model-style state tracking in text.
• Failure modes catalogued in §7.2 are today's open problems: retrieval degradation with memory growth, norm/constraint violations, and instruction-tuning personality bleed — the exact issues graph-memory papers (AriGraph, Zep, HippoRAG) cite as motivation.

Reading guide

• Read first: §4 (architecture: 4.1 memory+retrieval → 4.2 reflection → 4.3 planning/reacting) — ~4 pages containing everything durable. Then §6.5 for the ablation result and §7.1.2 for the emergent-behavior numbers.
• The one figure to study: Figure 7, the reflection tree for Klaus — it shows leaf observations becoming cited abstractions, the paper's deepest idea. Runner-up: Figure 6, the worked retrieval-score example (read the three columns of numbers against the formula).
• Worth reading for craft: §6.1–6.4 evaluation design (interview categories, TrueSkill over rankings, within-subjects ablations) — a reusable template for evaluating any agent architecture without a task benchmark.
• Skimmable: §2 related work, §3 (the Smallville tour and "day in the life" — charming, evidence-light), §5 implementation details unless you're rebuilding the sandbox.
• Do not skip: §7.2 (boundaries and errors) and §6.5.2 (hallucinated embellishments) — the honest failure catalogue is the part most secondary sources omit.