AgentBench

Full title: AgentBench: Evaluating LLMs as Agents Authors: Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, Jie Tang (Tsinghua University, The Ohio State University, UC Berkeley) Venue/year: ICLR 2024 — arXiv:2308.03688 (local PDF is v3, which adds later models such as claude-3 and glm-4 to the evaluation) Links: arXiv · local PDF: research-papers/2308.03688-agentbench.pdf

Why this paper matters

AgentBench is "the first systematic benchmark to evaluate LLM-as-Agent on a wide array of real-world challenges" (Figure 2 caption) — the moment agent evaluation stopped meaning "one demo environment" and started meaning a standardized, multi-environment battery with a unified harness. Arriving in August 2023, at the peak of AutoGPT/BabyAGI enthusiasm, it asked the sober question: across 8 interactive environments (5 built new for the paper) and 29 LLMs, who can actually act? Its two durable contributions: (1) the finding that agentic ability is a distinct, strongly capability-correlated axis — gpt-4 scores an overall 4.01 while the average open-weight model manages 0.51 (Figure 1b, Table 3) — and (2) the operational template for interactive LLM evaluation: POMDP task framing, CoT-formatted multi-turn dialogue, Docker-isolated environments behind an API-centric server-client toolkit, typed finish-reason taxonomy, and inverse-average-score task weighting (§2, §4.1). Later benchmarks (τ-bench included — it cites AgentBench as [14]) define themselves against this template.

The problem

Before AgentBench, evaluating "LLMs as agents" meant either text-game environments from the RL era (TextWorld, Jericho — "closed, discrete action spaces... primarily narrow focus on models' commonsense grounding," §1) or heavyweight multimodal simulators built for embodied agents, which "do not accurately reflect the practical use cases of LLMs" and can't evaluate text-only models (§1). Meanwhile standard LLM benchmarks (MMLU-style) test static single-turn knowledge, not "open-ended generation, multi-round interaction, and ability to act" (§5). Most agent papers evaluated on a single environment of their own design, making cross-model comparison impossible. Missing: a broad, text-only, practically grounded, interactive benchmark plus the engineering to run dozens of models through it reproducibly.

The mechanism

Formal framing (§2)

Interactive evaluation is defined as a POMDP (S, A, T, R, U, O) — state, action, transition, reward, task instruction space, observation — with the LLM agent as the policy. Deliberately, agents are run with the most primitive strategy: plain CoT prompting ("Thought" then "Action" in one output), no ensembles, no reflection, no search — "the easiest, cheapest, and most common way for people to deploy LLM agents" (§2). What is being measured is the model, not the scaffold.

The 8 environments, 3 groundings (§3, Table 2)

Code-grounded (§3.1):

• Operating System (OS) — real bash in Ubuntu Docker; deterministic QA ("how many users with non-/home directories?") and operational goals ("recursively set all files read-only, except mine"). Metric: success rate (SR).
• Database (DB) — full-pipeline SQL over authentic tables (select/insert/update). Metric: SR.
• Knowledge Graph (KG) — multi-hop QA over Freebase (45M+ entities, 3B facts) through tool-style query interfaces; partially observable by construction. Metric: answer F1.

Game-grounded (§3.2):

• Digital Card Game (DCG) — Aquawar, a turn-based team-battle game from the THU Agent Competition; strategy under explicit rules. Metric: win rate/reward.
• Lateral Thinking Puzzles (LTP) — riddles where the agent asks yes/no/irrelevant questions of an LTP host system; measures "depth and agility" of lateral reasoning. Metric: game progress.
• House-Holding (HH) — ALFWorld text-embodied household tasks ("put a pan on the dining table"). Metric: SR.

Web-grounded (§3.3):

• Web Shopping (WS) — WebShop e-commerce navigation, evaluated with pure prompting rather than the original trained models. Metric: reward.
• Web Browsing (WB) — Mind2Web cross-website tasks (click/select/type), adapted to prompted LLMs without fine-tuning. Metric: step SR.

The battery at a glance (Table 2)

The Weight⁻¹ row is each task's all-model average score — the normalizer used in the overall score (see the algorithm below). Its spread (3.5 for LTP vs 30.7 for WS) is the raw-score incomparability problem in a single row.

The finish-reason taxonomy (§2)

Every episode ends with a typed outcome — the benchmark's built-in failure diagnostics: Context Limit Exceeded (CLE), Invalid Format (IF — output doesn't follow the format instruction), Invalid Action (IA — format ok, action outside the action space), Task Limit Exceeded (TLE — round budget exhausted or repetitive generation), and Complete. "While IF and IA are mostly caused by LLMs' poor instruction following, TLE often indicates a weak multi-turn ability" (§2).

The algorithm

AgentBench's "algorithm" is its evaluation protocol — how interactive evaluation is actually operationalized (§4.1):

# --- Harness (server-client, API-centric; Appendix A) ---
# Each environment = a Docker image; each task subdivided into isolated workers.
# A model is "evaluated" by pointing the toolkit at any HTTP model server.

for task in 8_environments:                      # sizes/rounds fixed per Table 2
    for episode in task.test_split:              # Test: 1,014 episodes total (Dev: 269)
        history = [u_0]                          # u_0 = task instruction (+ 1-shot CoT demo)
        for round in 1 .. task.max_rounds:       # #Avg Round 5..35 per Table 2
            # context management: keep only what fits
            r = min r s.t. tokens(u_0, a_r, u_{r+1}, ..., u_k) <= 3500
            history_sent = [u_0 + "[NOTICE] 2r messages are omitted."] + tail
            a_k = LLM(history_sent, temperature=0)        # greedy, reproducible
            # a_k must contain "Thought: ..." + "Action: ..." in ONE round (CoT)
            outcome = env.execute(a_k)           # docker-isolated side effects
            if outcome in {IF, IA}: penalize/terminate per env rules
            if env.done: break                   # -> Complete | TLE | CLE
        record(score, finish_reason)

# --- Scoring: cross-task aggregation (§4.1 "Overall Score Calculation") ---
# Raw task scores are incomparable (WS rewards run high, LTP low), so:
weight[t]  = average score of ALL evaluated LLMs on task t     # Table 2 "Weight^-1"
OA(model)  = mean_t ( score[model, t] / weight[t] )            # overall AgentBench score

Walkthrough of the load-bearing choices:

• Dialogue-paradigm interaction: every environment is flattened into a user/agent chat (u_0, a_0, ..., u_k, a_k); environment feedback is "user" turns. Non-chat completion models get "USER:"/"AGENT:" prefixes appended (§4.1). This is what lets one harness drive 29 heterogeneous models.
• Truncation rule: history is trimmed to ≤3,500 tokens by dropping the middle rounds (keeping the instruction and the recent tail) and telling the model how many messages were omitted — a pragmatic memory policy that also caps cost (§4.1, footnote 2 gives the tokenizer-agnostic token estimate).
• Greedy decoding (temperature=0) for reproducibility across all tasks (§4.1) — note the contrast with τ-bench, which samples the user at temperature 1 precisely to study variance; AgentBench wants a deterministic capability snapshot.
• CoT in one round: "Thought" + "Action" emitted together per turn (format from ReAct), with a one-shot demonstration in the task instruction (§4.1).
• Weighting by the reciprocal of the all-model average score (the Weight^-1 row of Table 2: OS 10.8, DB 13.0, KG 13.9, DCG 12.0, LTP 3.5, HH 13.0, WS 30.7, WB 11.6): each task's average is rescaled to 1 so high-scoring tasks (WS) can't drown out hard ones (LTP). The weights are then frozen so future evaluations stay comparable (§4.1).
• Scale bookkeeping: Dev/Test = 269/1,014 episodes, "resulting in around 3k and 11k calls for inference, approximately the identical amounts of calls for inference as MMLU requires" — a deliberate cost envelope (§4.1).

Results that matter

gpt-4's own row (Table 3) is worth reading against the battery table above — even the top model is jagged: OS 42.4, DB 32.0, KG 58.8, DCG 74.5, LTP 16.6, HH 78.0, WS 61.1, WB 29.0. The two weak spots (lateral-thinking dialogue and real-website navigation) are long-horizon tasks with the least train-time-like structure.

Two more contours from §4.2–4.3:

• Every API-based commercial LLM clears OA 1.00 — "most of them can solve quite a few percent of problems" regardless of task — while most OSS models cluster far below (Table 3's three OSS blocks).
• Within OSS, size does not order agent skill (Figure 3): vicuna-13b (0.93) sits above 30B-class models and llama-2-70b; codellama-34b tops the scatter. Training data and alignment, not parameter count, dominate the OSS ranking.

Reading Table 4 (execution-outcome portions averaged across models) as a diagnosis matrix:

• TLE is the predominant failure cause overall — "revealing weak reasoning and decision-making abilities in LLMs" (§4.3): agents run out of rounds or loop, rather than crash.
• IF clusters where output syntax is strict (DB 53.3%, DCG 38.5% — SQL and structured game moves); IA clusters where the action space is easy to step outside (HH 64.1% — free-form household actions beyond the predefined set).
• Completed-portion extremes: OS 75.0% of episodes end normally vs LTP 14.0% and HH 13.1% — long-horizon game tasks are where interaction itself breaks down.

Limitations & how to defend the findings

• "Plain CoT under-measures models; better scaffolds would change the ranking." Deliberate scoping, argued in §2: without multiple trials or elaborate strategies, CoT is the cheapest, most common deployment mode — AgentBench measures the model floor, not the scaffold ceiling. A model ranking obtained under equal minimal scaffolding is exactly what's comparable.
• "The overall score depends on which models you averaged." True — weights are the reciprocal of the tested models' average per task (§4.1). The defense is in the same section: the weights are published and fixed "as a fixed weight for future overall score calculation," making OA a stable, if convention-bound, unit.
• "Greedy decoding hides variance." Yes — AgentBench reports a deterministic snapshot and cannot see the run-to-run inconsistency τ-bench later quantified with pass^k. Capability and reliability are different measurements; AgentBench only claims the former (temperature=0 "to ensure reproducible results," §4.1).
• "3,500-token truncation handicaps long-horizon tasks." Partly conceded by the data: CLE is rare (only 2,048-context text-davinci models hit it, §2), but TLE dominates failures (Table 4), and TLE conflates genuine reasoning failure with context starvation on 15–35-round tasks (KG, LTP, HH). The paper reads TLE as "weak multi-turn ability" (§2); a skeptic should note the truncation policy is part of what's being measured.
• "Environments have idiosyncratic metrics (F1, reward, progress, SR) — is averaging meaningful?" The weighting scheme (§4.1) normalizes scale but not meaning; OA is best read as a leaderboard statistic, with Table 3's per-environment columns as ground truth. The paper itself analyzes per-environment (§4.2–4.3) rather than leaning on OA alone.
• "Is the API-vs-OSS gap just 2023 vintage?" The v3 PDF partially answers this: newer models slot in where capability predicts (claude-3 at 3.11), and the paper's diagnosis — instruction following and multi-round consistency, improvable via "high quality multi-round alignment data" (abstract, §4.3 vicuna finding) — proved out as OSS models closed the gap in later years. The analysis aged better than the absolute numbers.
• Honest ambivalence the paper itself flags: code training is "a double-edged sword" — codellama beats llama-2 on procedural tasks (WS) but loses on ones needing general reasoning (DCG, OS) (§4.3).

Connections

• Capability-vs-reliability benchmarking: AgentBench is the capability pole — breadth across 8 environments, one deterministic run each; τ-bench (sibling page) is the reliability pole — 2 domains, many stochastic trials, pass^k. A serious agent evaluation needs both coordinates.
• ReAct (sibling shelf): AgentBench's per-turn "Thought + Action" format is ReAct operationalized as benchmark plumbing (§4.1 cites Yao et al. for the prompt organization) — evidence of how quickly ReAct became the default agent interface.
• τ-bench: beyond the philosophical contrast, τ-bench's function-calling agents vs AgentBench's text-format actions mark the 2023→2024 shift from prompt-parsed to native tool-calling; AgentBench's IF/IA failure classes are precisely what native function calling later suppressed.
• SWE-bench / WebArena: the depth-first counterparts — one realistic domain, maximal fidelity — versus AgentBench's breadth-first battery; the field now uses AgentBench-style batteries for model cards and domain benchmarks for deployment decisions.
• Scaling and alignment lore: two of the paper's side findings became standard citations — vicuna-13b matching a 3×-larger code model (alignment data quality > size, §4.3) and llama-2-70b ≈ llama-2-13b (under-trained scale is wasted scale, read via Hoffmann et al.'s law, §4.3).
• MT-Bench (sibling shelf): both confront "how to score heterogeneous open-ended behavior"; MT-Bench answers with LLM judges, AgentBench with environment-verifiable outcomes plus score normalization — the two dominant grading philosophies for post-static-benchmark evaluation.

Reading guide

• Read first: Abstract → Figure 1 (the radar + bar chart is the paper in one image) → §2 (POMDP + finish reasons, one page) → §4.1 (the evaluation protocol — the operational core) → Table 3.
• Then: §4.3 (failure-portion analysis, code-training ambivalence, alignment findings — the most durable insights) with Table 4 and Figure 3.
• Skimmable: §3 (environment blurbs — skim all eight in five minutes; depth lives in Appendices B–I), §5 (related work), §4.2 (reads Table 3 aloud).
• Appendices, only as needed: A (toolkit architecture: Docker images, task workers, HTTP model serving), B–I (per-environment construction and prompts — go here only when reproducing a specific environment), J (extended analysis and case studies; J.2 has planning/self-correction/tool-use case studies).
• The one table to study: Table 3 — the full 29-model × 8-environment matrix with OA. Read it column-wise once (which environments discriminate models?) and row-wise once (where does gpt-4 fail? LTP at 16.6, WB at 29.0 — long-horizon lateral reasoning and real-web grounding), and cross-check with Table 4's failure types; the three views together are the whole diagnosis.

If you plan to run it rather than read it: the released package (github.com/THUDM/AgentBench) is the artifact — Docker images per environment, an HTTP model-server contract, and task workers; §4.1's "Toolkit" paragraph plus Appendix A are the operating manual, and everything above about truncation, temperature, and weights is what the harness enforces for you.