Self-Consistency

Full title: Self-Consistency Improves Chain of Thought Reasoning in Language Models Authors: Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou (Google Research, Brain Team) Venue/year: ICLR 2023 (arXiv v4, March 2023) arXiv: https://arxiv.org/abs/2203.11171 Local PDF: /Users/moscalej/Documents/Onep/research-papers/2203.11171-self-consistency.pdf

Why this paper matters

Self-consistency (CoT-SC) is the simplest and most durable test-time-compute result in the LLM reasoning literature: replace greedy decoding with sample k diverse chains of thought, then majority-vote the final answers. Nothing is trained, no verifier or re-ranker is added, no human annotation is collected — "entirely unsupervised … off-the-shelf" (§1) — yet it delivered +17.9% absolute on GSM8K over CoT prompting and new SOTA on most arithmetic/commonsense benchmarks of its day (abstract, Table 2). Historically it sits between chain-of-thought (Wei et al., 2022), which it upgrades, and everything after: ToT uses it as a baseline and generalizes it to step-level voting, ReAct combines with it for internal/external knowledge routing, and "majority voting over k samples" remains the default cheap accuracy lever and uncertainty signal in agent systems today.

The problem

CoT prompting was paired with greedy decoding: one deterministic reasoning path per question (§1). That has three defects the paper attacks head-on:

• A single path can be locally optimal but wrong — greedy decoding suffers from "repetitiveness and local-optimality" while one stochastic sample suffers high variance (§1).
• Complex problems "typically admit multiple different ways of thinking leading to its unique correct answer" (abstract); a single decode cannot exploit that structure.
• Prior fixes were heavy: train an added verifier (Cobbe et al., 2021) or train a re-ranker with human annotations — extra models, extra supervision (§1).

The missing observation: wrong reasoning paths are diverse in their wrongness, while correct paths agree — "correct reasoning processes, even if they are diverse, tend to have greater agreement in their final answer than incorrect processes" (§2). Agreement is therefore a signal you can harvest with sampling alone.

The mechanism

Sample-and-marginalize decoding (§2)

Three steps (Figure 1): (1) prompt with manual CoT exemplars; (2) instead of greedy decoding, sample k reasoning paths from the decoder (temperature / top-k / nucleus — the method is agnostic, §2, §3.5); (3) marginalize out the reasoning by taking the most common final answer.

The latent-variable view is the paper's key formal move. Each sampled output i is a pair (r_i, a_i): r_i is the reasoning path (a token sequence, treated as a latent variable) and a_i ∈ A is the parsed final answer, where generating r_i → a_i means the path is "only used to reach the final answer" (§2). The answer distribution you actually care about is the marginal over paths, and the vote is its plug-in estimate.

The aggregation math (§2, Eq. 1, Table 1)

Majority vote (unweighted sum) — the method:

a* = argmax_a  Σ_{i=1..m} 1(a_i = a)

m = number of sampled outputs; 1(·) = indicator; a* = the "most consistent" answer.

Weighted variants — weight each output by its model likelihood P(r_i, a_i | prompt, question), either unnormalized (product of token probabilities) or length-normalized (Eq. 1):

P(r_i, a_i | prompt, question) = exp( (1/K) Σ_{k=1..K} log P(t_k | prompt, question, t_1..t_{k-1}) )

t_k = k-th token of the output (r_i, a_i); K = its total token count; the (1/K) factor is the length normalization (log-prob averaged per token, then exponentiated).

The empirical punchline (Table 1, PaLM-540B): the plain unweighted vote (GSM8K 74.4) matches the normalized weighted sum (74.1) — because the model's normalized probabilities for different generations "are quite close to each other," i.e. the LM is poorly calibrated over its own reasoning paths (§2, footnote 2). So the simplest rule is also the right one; weighting by confidence buys nothing, and "weighted average" (each answer's weighted sum divided by its count) is much worse (GSM8K drops to 22.1 with the normalized weighted average). Applicability caveat, stated by the authors: the vote needs a fixed answer set to count over; extending to free-form generation requires a consistency metric between generations (§2 end / §3 intro).

Experimental scope (§3.1)

Four model families — UL2-20B, GPT-3 175B (code-davinci-001/002), LaMDA-137B, PaLM-540B — same fixed CoT prompts as Wei et al. (2022). Sampling: T=0.5 with top-k 40 (UL2, LaMDA), T=0.7 with top-k 40 (PaLM-540B), T=0.7 no top-k (GPT-3). Main results sample 40 paths, averaged over 10 runs; std ≤ 0.5 throughout (footnote 7).

The algorithm

# SelfConsistency(question, cot_prompt, LM, m, parse, sampler_params)
answers ← []                                   # multiset of final answers
for i = 1 .. m:                                # m = 40 in the paper's main results
    (r_i, a_i) ← LM.sample(cot_prompt + question; T, top_k)   # one full CoT + answer
    a_i ← parse(r_i output)                    # e.g. first number after "The answer is"
    answers.append(a_i)
return argmax_a count(answers, a)              # majority vote = marginalize out r_i

Line-by-line:

1. cot_prompt — unchanged few-shot CoT exemplars (8 manual exemplars for arithmetic, 4–7 for commonsense; §3.1). Self-consistency modifies decoding only; it composes with any prompt, including zero-shot CoT (Table 8: 43.0 → 69.2 on GSM8K, +26.2%).
2. LM.sample — diversity is the active ingredient. Any standard sampling scheme works and results are robust across T, k, p (§3.5, Figure 4 left). Contrast with beam search: on UL2-20B, self-consistency over beam-searched paths underperforms self-consistency over sampled paths (Table 6) because "beam search yields a lower diversity in the outputs" (§3.4).
3. parse — answers must be extracted into a fixed answer set A (task-dependent parser; footnote 1: e.g. the first numerical part after "The answer is").
4. Vote — the argmax above; ties are simply whatever count breaks. No weights (Table 1 justifies), no learned components.
5. Cost dial — accuracy rises monotonically and saturates with m (Figure 2); "people can try a small number of paths (e.g., 5 or 10) as a starting point to realize most of the gains" (§5). The vote margin doubles as an uncertainty estimate: % of decodes agreeing with the winner correlates strongly with accuracy (Figure 5) — "self-consistency confers some ability for the model to 'know when it doesn't know'" (§3.5).

Results that matter

CoT-prompting (greedy) → self-consistency (40 paths); all numbers are the paper's own.

At the time, this set new SOTA on GSM8K, SVAMP, AQuA, StrategyQA and ARC-c — beating even GPT-3 fine-tuned on 7.5k examples plus a trained verifier on GSM8K (Table 2 header notes, §3.2). Comparisons that isolate the mechanism: sample-and-rank with equal samples gains far less (Figure 3); prompt-order and multi-prompt ensembles gain far less (Table 7: GSM8K 17.1 → 18.6/ 19.2 vs 27.7 for SC on LaMDA-137B); gains grow with scale — +3–6% for UL2-20B vs +9–23% for LaMDA-137B/GPT-3 (§3.2).

Limitations & how to defend the findings

• "It's a 40x compute increase." Conceded as the first limitation (§5): "it incurs more computation cost." Defense in the data: Figure 2 shows steep early saturation — most of the gain arrives by 5–10 paths, and the paper recommends exactly that. The knob is also graceful: any m improves on greedy (Table 6/Figure 2 show gains from m=5 on).
• "Voting only works because the answer space is small — is it just guessing-noise reduction?" Two rebuttals from the paper. First, applicability is restricted to fixed answer sets, stated plainly (§2 end). Second, the gains are not from answer-space collapse: sample-and-rank with the same samples (Figure 3) and prompt ensembles (Table 7) vote/rank over the same spaces and gain much less — the reasoning-path diversity plus marginalization is the mechanism, exactly the latent-variable story of §2.
• "Does a majority of wrong paths ever win?" Yes — the vote helps only when correct answers agree more than errors correlate. The paper documents the failure edge: "language models can sometimes generate incorrect or nonsensical reasoning paths (e.g., the StrategyQA example in Table 4, the two population numbers are not exactly correct), and further work is needed to better ground models' rationale generations" (§5). Also note Table 4's flip side: a correct final answer can ride on imperfect facts, so voting validates answers, not rationales.
• "Is it sensitive to the sampling recipe or prompt quality?" Ablated: robust across temperature/top-k/nucleus settings (Figure 4 left, §3.5); it repairs imperfect prompts (Table 8: greedy drops 17.1 → 14.9, SC lifts to 23.4) and works with zero-shot CoT (+26.2%).
• "Small models?" Gains are "relatively lower for smaller models" (§3.5, Figure 4 right) — UL2-20B sees +3–6%; the method amplifies abilities that "only emerge when the model reaches a sufficient scale."
• "Why not weight votes by model confidence?" Tried; doesn't help (Table 1). The stated reason — near-identical normalized sequence probabilities — is itself a finding: the LM "cannot distinguish well between correct solutions and wrong solutions," which is why prior work needed trained re-rankers (footnote 2).

Connections

• Chain-of-thought (Wei et al., 2022) — self-consistency is a strict drop-in upgrade of CoT's decoder; same prompts, no training. It also makes CoT safe where it used to hurt: on ANLI/e-SNLI/RTE, CoT underperformed standard prompting and SC reverses that (Table 5, §3.3).
• Voting math and correlated errors — the §2 latent-variable framing is a marginalization P(a) = Σ_r P(r, a) estimated by Monte-Carlo; the vote wins precisely while errors stay decorrelated across sampled paths. Beam search (Table 6) is the in-paper demonstration of what correlated samples do to the estimate — diversity is load-bearing.
• ReAct — uses CoT-SC (21 paths) as its strong internal-knowledge baseline, and its best HotpotQA/FEVER systems route between ReAct and CoT-SC based on vote confidence — an early production use of SC's uncertainty signal (see react.md).
• Tree of Thoughts — formalizes CoT-SC as a special-case tree (k chains, vote at the leaves only) and lifts the vote to intermediate states: ToT's vote-evaluator is "step-wise self-consistency" (see tree-of-thoughts.md).
• Reflexion — the orthogonal axis of test-time compute: SC spends samples in parallel with no information flow between them; Reflexion spends trials sequentially with verbal memory between attempts (see reflexion.md).
• Verifiers / re-rankers (Cobbe et al., 2021) — the trained-supervision alternative SC displaced on GSM8K; the calibration finding (footnote 2) explains why those needed training in the first place, and why later process-reward-model work re-introduced learned scoring.

Reading guide

• Read first: abstract, §1, and Figure 1 — the entire method is the figure; then §2 carefully: the latent-variable formulation, Eq. 1, and the Table 1 aggregation comparison are the paper's intellectual content.
• Then: §3.2 with Tables 2–3 (headline results; check the "Previous SoTA" rows), and Figure 2 (the accuracy-vs-#paths curves you will reproduce mentally every time you set k).
• Then: §3.4 (Figure 3, Tables 6–7) — the comparisons that prove the mechanism is diverse sampling + marginalization, not generic ensembling.
• Skim: §3.3 (Table 5, CoT-hurts cases), §3.5 (robustness; Figure 5's consistency-accuracy correlation is the practical gem), §4 Related Work.
• Appendices as needed: A.1.2 (prompt-set robustness), A.1.3 (model-ensemble comparison), A.3 (full prompts).
• The one figure to study: Figure 1 — greedy decode vs three sampled paths with the vote — plus Table 1 if you want the one table that justifies "just count answers."