Multi-Agent Orchestration Patterns

The cheapest, most reliable agent is the one you don't build. Anthropic's guidance opens with exactly this warning: find the simplest solution possible, and only increase complexity when it demonstrably improves outcomes. A single agent-loop with good tools handles a surprising range of work, and every additional agent you add multiplies tokens, latency, and the number of places a run can go wrong. So before reaching for a topology, the real question is whether you need more than one agent at all.

You usually do not when the task decomposes into fixed steps (use a code-orchestrated workflow — prompt chaining, routing, parallelization — rather than autonomous agents at all), or when all the work shares one context that must stay coherent. You do when the problem is open-ended and its subtasks can't be predicted in advance, when separating concerns into focused prompts and toolsets improves accuracy, or when independent sub-problems can run concurrently to cut wall-clock time. The rest of this page assumes you've cleared that bar and now have to choose how the agents relate to one another.

Single-agent vs multi-agent: the core tradeoff

A multi-agent system splits a problem across several agents, each with its own instructions, tools, and often its own context window, coordinated by some control structure. The upside is specialization and parallelism; the downside is coordination cost. Anthropic's own multi-agent research system makes the price concrete: agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats, and in their evaluation token usage by itself explains 80% of the variance in performance. The practical rule they draw from this is blunt — multi-agent architectures only pay off on high-value tasks that involve heavy parallelization, information exceeding a single context window, and many complex tools. They underperform when all agents must share the same context or when there are many dependencies between agents; most coding tasks fall in this bucket because they have fewer truly parallelizable sub-tasks than research, and current models are still weak at coordinating and delegating in real time.

Keep two orthogonal axes in mind as you read the topologies below, because they explain most of the differences:

• Where control lives — centralized (a coordinator decides who runs next) vs decentralized (agents transfer control directly to one another).
• How agents communicate — through shared state everyone can read, or through scoped messages / handoffs that pass only what the next agent needs.

flowchart TD
    subgraph Centralized
        S[Supervisor] --> A1[Worker A]
        S --> A2[Worker B]
        A1 -.results.-> S
        A2 -.results.-> S
    end
    subgraph Decentralized
        H1[Agent A] -->|handoff| H2[Agent B]
        H2 -->|handoff| H3[Agent C]
        H3 -->|handoff| H1
    end

Supervisor / orchestrator-worker

How it works. A single coordinating agent — the supervisor (Anthropic and the orchestrator-worker pattern call it the orchestrator or lead agent) — owns the goal. It does not do the domain work itself; it analyzes the request, decides which specialist should act next, delegates a scoped subtask, and incorporates each result before deciding the next move. In Anthropic's framing of the orchestrator-worker workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results. The distinguishing feature versus plain parallelization is that the subtasks are not predefined — the orchestrator invents them based on the specific input, which is why it suits complex tasks where you can't predict the subtasks in advance.

In LangGraph this is the agent supervisor architecture: the supervisor is essentially "an agent whose tools are other agents", each worker keeps its own scratchpad and only appends its final result to shared state. CrewAI exposes the same idea as Process.hierarchical, where you supply a manager_agent (or a manager_llm from which one is auto-created) that handles planning, delegation, and validation — tasks are not pre-assigned, the manager allocates them to agents by capability and reviews their outputs. The OpenAI Agents SDK reaches the supervisor shape through agents-as-tools: a manager agent keeps control of the conversation and calls specialists via Agent.as_tool(), which is the right choice when "you want one agent to own the final answer, combine outputs from multiple specialists, or enforce shared guardrails in one place."

flowchart TD
    U[User goal] --> O[Supervisor / Orchestrator]
    O -->|delegate subtask 1| W1[Worker: Search]
    O -->|delegate subtask 2| W2[Worker: Analysis]
    O -->|delegate subtask 3| W3[Worker: Code]
    W1 -->|result| O
    W2 -->|result| O
    W3 -->|result| O
    O --> SYN[Synthesize final answer]
    SYN --> U

When to use it. Open-ended tasks whose decomposition can't be predicted; work that benefits from a single point that owns the final synthesis and shared guardrails; situations where you want each worker's context kept small and focused.

Tradeoffs. Centralized control is easy to reason about and debug — there is one place that decides everything — but the supervisor is a bottleneck and a single point of failure, and every delegation round adds a full LLM turn. Anthropic spins up 3–5 subagents in parallel in their research system (and has each subagent call 3+ tools in parallel), which cut research time by up to 90% on complex queries; the flip side is the 15×-tokens cost noted above.

Failure modes. The supervisor over-decomposes (Anthropic saw early versions spawn 50 subagents for simple queries), workers duplicate each other's work because the orchestrator's instructions were vague, or the synthesis step drops information the workers found. Synchronous execution means a slow worker stalls the whole batch.

Cost/latency. Highest per-task token cost of the centralized patterns because of repeated orchestrator turns plus worker context, partly offset by parallel worker execution shrinking wall-clock time. Reserve it for tasks valuable enough to justify ~15× chat token spend.

Hierarchical (supervisor-of-supervisors)

How it works. When a single supervisor's span of control gets too wide, you nest them: a top-level supervisor coordinates mid-level supervisors, each of which manages its own team of workers. LangGraph implements this directly as hierarchical agent teams — the subagents are themselves LangGraph objects (graphs), so a node in the parent graph is an entire team (managed by a supervisor agent that connects them), and you get recursive delegation. LangGraph's own multi-agent taxonomy places this hierarchical shape — which it characterizes as a "supervisor of supervisors" — at the far end of a spectrum that runs from the loosely-coupled network pattern, through the single supervisor, to these nested teams.

When to use it. Large systems where dozens of specialists would overwhelm one router, or where the domain naturally splits into teams (a "research" team, an "ops" team) each with internal structure. The extra layer lets each supervisor reason over a manageable handful of children.

Tradeoffs. It scales the supervisor pattern past its fan-out limit and keeps each level's prompt focused, at the cost of more orchestration hops — every additional layer adds at least one more LLM turn to the critical path, compounding latency and token cost. Tracing a decision now means walking a tree rather than a single hub.

Failure modes. Goal drift as intent is paraphrased down each layer; latency stacking when layers execute serially; and ambiguous ownership when a subtask could belong to two teams.

Cost/latency. The most expensive centralized shape — token cost and latency grow roughly with depth. Justified only when the breadth of specialists genuinely can't be flattened.

Sequential / pipeline

How it works. Agents (or steps) are chained so each one's output becomes the next one's input, in a fixed order. This is Anthropic's prompt chaining workflow — decompose a task into sequential steps, optionally with programmatic gates between them that check the intermediate result before proceeding. CrewAI's default Process.sequential is the same shape: tasks execute in the order listed, and the output of one task serves as context for the next.

When to use it. When the task cleanly decomposes into fixed, ordered subtasks and you'd rather trade some latency for accuracy — e.g. draft, then translate, then fact-check. Because the path is predefined, this is a workflow, not an autonomous agent system, and it's the simplest thing that can possibly work for ordered decomposition.

Tradeoffs. Maximally predictable and debuggable; trivially observable because data flows one way. But it is strictly serial — total latency is the sum of every stage — and inflexible: it can't adapt the step sequence to the input the way an orchestrator can. Gates let you fail fast but add checkpoints to maintain.

Failure modes. Error propagation — a mistake early in the chain is faithfully carried to the end unless a gate catches it — and brittleness when an input doesn't fit the fixed pipeline.

Cost/latency. Predictable and moderate: one LLM call per stage, no parallelism, latency adds up linearly with stage count.

Parallel (sectioning and voting)

How it works. Independent subtasks run simultaneously and a programmatic step aggregates their outputs. Anthropic distinguishes two variants of parallelization: sectioning breaks a task into independent subtasks run in parallel (e.g. one model instance answers while another screens the same input for policy violations as a guardrail), and voting runs the same task multiple times to get diverse outputs for higher confidence (e.g. several instances independently review code for a vulnerability, and you take the union or a majority). The key contrast with orchestrator-worker is that here the subtasks are fixed and known up front, so no LLM has to plan the split.

When to use it. When subtasks are genuinely independent and you want speed (sectioning), or when multiple attempts/perspectives raise confidence on a single hard judgment (voting).

Tradeoffs. Cuts wall-clock latency dramatically when subtasks are independent, and voting improves reliability — but it spends tokens on N parallel runs, and aggregation logic (how to merge or vote) is your responsibility and can itself be a source of bugs.

Failure modes. Falsely assuming independence (subtasks that actually share state produce inconsistent merges); voting that masks a systematic bias all instances share; and aggregation that silently drops minority-but-correct answers.

Cost/latency. Low latency, high peak concurrency cost. Total tokens scale with the number of parallel branches; latency is bounded by the slowest branch rather than their sum.

Swarm / handoff (decentralized control transfer)

How it works. There is no central coordinator. Each agent can hand off control directly to another agent, and after a handoff the receiving agent becomes the active one for the rest of the turn. The mechanism that made this concrete is OpenAI's experimental, educational, stateless Swarm framework: if a function returns an Agent, execution will be transferred to that Agent — a handoff is just a tool/function whose return value is the next agent. The OpenAI Agents SDK productizes this: handoffs are represented as tools to the LLM, so a handoff to a "Refund Agent" surfaces to the model as a tool named transfer_to_refund_agent (the default from Handoff.default_tool_name()). You build them with the handoff() function, customizing tool_name_override, tool_description_override, an on_handoff callback, an input_type schema for small model-generated metadata (e.g. a reason or priority), and an input_filter that controls what conversation history the receiving agent sees. AutoGen's Handoffs design pattern is explicitly "introduced by OpenAI in an experimental project called Swarm" and is decentralized: there is no central orchestrator — agents delegate via special handoff tools over event-driven pub/sub, each independently deciding whether to handle or transfer.

LangGraph expresses the same control transfer through its Command primitive: a node returns a Command(goto=..., update=...) object that specifies not only the update to the state but also which node to go to next. For handoffs that cross subgraph boundaries you set graph=Command.PARENT so the jump targets a node in the parent graph. This is also how LangGraph builds its network architecture, where in principle every agent can route to every other agent.

sequenceDiagram
    participant U as User
    participant T as Triage Agent
    participant B as Billing Agent
    participant R as Refund Agent
    U->>T: "I was charged twice"
    Note over T: calls transfer_to_billing_agent
    T->>B: handoff (control transfers)
    Note over B: determines a refund is owed,
calls transfer_to_refund_agent
    B->>R: handoff (control transfers)
    R->>U: issues refund, responds directly

When to use it. Conversational routing where the chosen specialist should own the rest of the interaction and respond directly to the user, where you want each agent's prompt kept tightly focused, and where the routing decision is itself part of the workflow. The OpenAI SDK's own heuristic: prefer handoffs when "routing itself is part of the workflow and you want the chosen specialist to own the next part of the interaction," and prefer agents-as-tools when a specialist should help with a bounded subtask but not take over the user-facing conversation.

Tradeoffs. Decentralization scales well — adding a specialist is local, you just give peers a handoff to it — and keeps prompts small. But there is no single place that sees the whole picture, so behavior is harder to trace and reason about, and input_filter choices (what history the next agent inherits) become correctness-critical.

Failure modes. Handoff loops (A → B → A …) with no agent owning termination; context loss when the filter strips something the receiver needed; and "hot-potato" routing where agents keep passing a request none will own.

Cost/latency. Generally lighter than supervisor patterns for routing because control moves laterally instead of returning to a hub each time — but unbounded handoff chains can blow up both latency and token cost, so loop guards matter.

Blackboard / shared-state

How it works. Agents don't message each other at all; they collaborate indirectly through a shared, structured workspace — the blackboard. The classic pattern (from the 1970s Hearsay-II speech system) has three parts: the blackboard holding the evolving problem state and partial solutions, a set of knowledge sources (the specialist agents) that each read the board and write contributions in their area of expertise, and a control component that decides which knowledge source acts next based on the board's current state. Because communication is mediated entirely by the shared structure, agents are decoupled — you can add or remove a specialist without touching the others. (In LLM systems this is often realized as shared memory or a shared scratchpad; note that LangGraph's collaboration mode, where agents share a single message scratchpad, is a close cousin — powerful but, as its own docs warn, prone to becoming "overly verbose.")

When to use it. Problems whose solution order isn't known in advance and where many specialists have complex, interdependent contributions — when the dependency graph "looks more like a web than a chain."

Tradeoffs. Maximum flexibility and loose coupling; new specialists slot in cleanly. But the shared state becomes a coordination hotspot — concurrent writes need a conflict policy — and a fat, fully-visible board inflates every agent's context (and token bill). Determinism suffers because which contribution lands first can change the outcome.

Failure modes. Context bloat from an ever-growing board; non-termination if no control rule recognizes "done"; and write conflicts / thrashing when several agents revise the same region.

Cost/latency. Token cost grows with board size since agents re-read shared context each turn; latency depends on the control regime (serial control = slow, opportunistic concurrency = faster but contention-prone).

Router / dispatcher

How it works. A lightweight first step classifies the input and dispatches it to the appropriate downstream handler — Anthropic's routing workflow: classify an input and direct it to a specialized follow-up task. The router itself typically doesn't solve the problem; it picks the handler. A common use is sending easy queries to a small, cheap model and hard ones to a larger model. A router differs from a supervisor in that it makes a single up-front classification and then steps out of the way, whereas a supervisor stays in the loop across many delegation rounds. (CrewAI's manager and AutoGen's group-chat manager that selects the next speaker are richer, iterative relatives of this idea.)

When to use it. Inputs that fall into distinct categories better handled by specialized prompts, when classification can be done accurately, and when you want to route by cost/complexity (cheap model vs expensive model).

Tradeoffs. Cheap and simple — one classification call, then a focused handler — and it isolates concerns so each handler's prompt stays clean. But the whole system's correctness now hinges on the classifier; a misroute sends the request to a handler that can't recover it.

Failure modes. Misclassification (no path back if there's no fallback), and category creep where real inputs don't fit any defined bucket.

Cost/latency. Minimal overhead — a single, often small-model, classification step — and it can reduce total cost by steering most traffic to cheaper handlers.

Group chat (a centralized relative worth naming)

Several frameworks offer a group chat shape that sits between supervisor and blackboard: agents share the full conversation and a manager picks who speaks next. AutoGen models this as a star topology with a central GroupChatManager — all participants subscribe and publish to one common topic, a participant publishes a GroupChatMessage to that shared topic, and the manager selects the next speaker and sends it a RequestToSpeak message. AutoGen's core design-pattern doc notes that "typically, a round-robin algorithm or a selector with an LLM model is used" to choose the speaker — it does not name a default. (Heads-up on a common conflation: the "auto" speaker-selection label belongs to the older AutoGen 0.2 AgentChat GroupChat API, whose speaker_selection_method defaults to "auto" — an LLM-based selector — with "manual", "random", and "round_robin" as the other built-in options; the core pattern above is a separate, lower-level implementation.) It gives iterative refinement and full shared context (like a blackboard) but with centralized turn-taking (like a supervisor) — at the cost of every agent paying for the entire transcript in its context.

Choosing a topology

flowchart TD
    Q{Can the task be done
by one agent + tools?} -->|Yes| One[Single agent — stop here]
    Q -->|No| Fixed{Are the subtasks
fixed & ordered?}
    Fixed -->|Yes, ordered| Pipe[Sequential / pipeline]
    Fixed -->|Yes, independent| Par[Parallel: sectioning / voting]
    Fixed -->|No, must be planned| Own{Should one agent own
the final synthesis?}
    Own -->|Yes| Sup[Supervisor / orchestrator-worker]
    Sup -->|too many specialists| Hier[Hierarchical teams]
    Own -->|No, specialist responds directly| Sw[Swarm / handoff]
    Own -->|Many interdependent specialists,
order unknown| BB[Blackboard / shared-state]
    Q -->|Just pick a handler| Rt[Router / dispatcher]

The patterns are not exclusive — real systems compose them (a router in front of a supervisor whose workers each run a pipeline). The discipline Anthropic urges throughout applies at every layer: add a coordination mechanism only when a simpler one demonstrably falls short, because each one you add is paid for in tokens, latency, and the difficulty of tracing what went wrong. For the wire-level protocols that let these agents actually talk across process and organizational boundaries — A2A between agents and MCP to tools — see protocols-mcp.html, protocols-a2a.html, and the side-by-side in protocols-comparison.html. For concrete framework choices, see ecosystem-frameworks.html, and for running these systems in production, ecosystem-ops.html.