Agents & the Agent Loop

Agent vs. agentic system

It is worth being precise about two words that get used interchangeably. An agent is a single language-model-driven actor that pursues a goal by using tools in a loop, deciding for itself what to do next based on feedback from its environment. An agentic system is the larger software around it — possibly several agents, plus the orchestration code, memory, tools, and guardrails that connect them to the real world.

Anthropic's Building Effective Agents draws a sharper line inside that space. It separates workflows, "systems where LLMs and tools are orchestrated through predefined code paths," from agents, "systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks." The difference is who decides the next step: in a workflow the control flow is written by the engineer; in an agent the model chooses it at runtime. That single distinction — fixed code paths vs. a model-driven loop — is the dividing line this whole site is organized around.

The agent loop

The defining structure of an agent is the agent-loop: a cycle of observe → reason → act → observe … that repeats until a stopping condition is met. Anthropic describes the canonical agent as exactly this — an LLM "using tools based on environmental feedback in a loop," gaining "ground truth from the environment at each step (such as tool call results or code execution)."

Each turn of the loop has three moving parts:

• Observe. The model is given the current context: the goal, the history of prior actions and their results, and any new observation returned by the environment.
• Reason. The model is the controller — it inspects that context and decides what to do next. It may reason in natural language about the situation before committing to anything.
• Act. The model emits an action, almost always a structured tool-use call: a named capability invoked with structured arguments (read a file, query a database, call an API). The environment executes it and returns an observation, which becomes the input to the next turn.

The LLM here is not the whole program; it is the policy that picks the next action. The action space — the set of things the agent can do — is exactly the set of tools it has been given. This is why tool design dominates agent quality: the tools are the agent's vocabulary for affecting the world, and a result that is fed back clearly is a result the model can reason about on the next turn.

flowchart TD
    Start([Goal / task]) --> Observe[Observe: assemble context
goal + history + latest observation]
    Observe --> Reason{Reason: controller
decides next action}
    Reason -->|emit tool call| Act[Act: execute tool
with structured arguments]
    Act --> Result[Environment returns
observation / result]
    Result --> Stop{Stopping
condition?}
    Stop -->|no| Observe
    Stop -->|task complete| Done([Return answer])
    Stop -->|max iterations / error budget| Halt([Halt or escalate])
    Reason -->|answer directly| Done

Reasoning patterns

The loop above is a skeleton. What an agent does inside the "reason" step is where the major patterns differ.

ReAct: interleaved reasoning and acting

ReAct (Yao et al., 2022/2023) is the foundational pattern: it generates "both reasoning traces and task-specific actions in an interleaved manner." The key formal move is to augment the action space. ReAct lets the model emit, in addition to real actions, a special "action" that is just free-form language — a thought, or reasoning trace, that "does not affect the external environment" and produces no observation. The paper writes this as expanding the action space  = A ∪ L, where L is the space of language. Thoughts let the model "induce, track, and update action plans as well as handle exceptions," while real actions let it pull in external information.

In the paper's knowledge-task setup, the real action space is a simple Wikipedia API with three actions:

• search[entity] — returns the first 5 sentences of the entity's wiki page, or suggests similar entities if it doesn't exist;
• lookup[string] — returns the next sentence on the page containing string (a Ctrl+F);
• finish[answer] — ends the task with answer.

A ReAct trajectory therefore reads as an alternating transcript of thoughts, actions, and observations:

Thought 1: I need to find when the Apollo program ended, so I'll search for it.
Action 1:  search[Apollo program]
Obs 1:     The Apollo program ... ran from 1961 to 1972 ...
Thought 2: The program ended in 1972. I can answer now.
Action 2:  finish[1972]

Because the model can interleave grounded lookups with reasoning, ReAct "overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning" — pure chain-of-thought has no way to check itself against the world, so an early wrong fact poisons the rest of the chain. Empirically, on the interactive benchmarks ALFWorld and WebShop, ReAct beat imitation- and reinforcement-learning baselines by an absolute success rate of 34% and 10% respectively, using only one or two in-context examples; its best ALFWorld trial reached a 71% success rate. ReAct is the conceptual default for a tool-using agent and maps directly onto the loop diagram above, with the "reason" node emitting either a thought or an action each turn.

Planning and plan-and-execute

ReAct decides one step at a time. A planning agent instead commits to a multi-step plan up front, then executes the steps — a plan-and-execute shape. The primary motivation comes from Plan-and-Solve prompting (Wang et al., 2023): replacing chain-of-thought's "Let's think step by step" with "Let's first understand the problem and devise a plan to solve the problem. Then, let's carry out the plan and solve the problem step by step." Devising the plan first reduces "missing-step" errors by forcing the model to lay out the whole task before getting lost in any one part. (LangChain's "Plan-and-Execute" agent is a direct descendant of this idea.)

The trade-off is explicit: a plan made up front gives global structure and lets you parallelize or audit steps, but it is brittle if reality diverges from the plan. Practical planners therefore re-plan — they treat the plan as a living document, revising it whenever an observation invalidates it, which folds planning back into the outer loop rather than running once before it.

Reflection and self-critique

A reflection agent adds a self-critique step: after an attempt, the model evaluates its own output and feeds that critique back as additional context for the next attempt. Reflexion (Shinn et al., 2023) formalizes this as "verbal reinforcement" — instead of updating model weights, the agent generates natural-language reflections on failure and stores them to do better next time. Reflexion is explicitly not reinforcement learning by gradient: it "reinforce[s] language agents not by updating weights, but instead through linguistic feedback," which makes it lightweight and free of fine-tuning.

Reflexion is built from three distinct models:

• an Actor (M_a) that generates text and actions (often itself a ReAct or chain-of-thought agent);
• an Evaluator (M_e) that scores the Actor's trajectory (via environment signal, heuristics, or self-evaluation such as self-written unit tests);
• a Self-Reflection model (M_sr) that turns a failed trajectory plus its score into a written natural-language reflection.

These are tied together by two kinds of memory: short-term memory is the current trajectory, while long-term memory is a buffer of reflections from past trials that is prepended to the context on the next attempt. Reflexion reported a 91% pass@1 on the HumanEval coding benchmark (vs. 80% for the GPT-4 baseline at the time), improved AlfWorld decision-making by an absolute 22% over baselines within 12 learning steps, and improved HotPotQA reasoning by 20%; combined with ReAct it completed 130 of 134 AlfWorld tasks. These three patterns compose: a strong agent is often a ReAct Actor that re-plans and reflects between attempts.

Stopping conditions and error handling

A loop that can act indefinitely needs a way to halt. There are two healthy reasons to stop and several unhealthy ones to guard against:

• Task completion. The model decides the goal is met and emits a terminal action (ReAct's finish[...], or a tool-free final message). This is the intended exit.
• Resource limits. Anthropic recommends "stopping conditions (such as a maximum number of iterations) to maintain control," so a confused agent can't loop forever. A step budget, a token/cost budget, or a wall-clock timeout are all common.
• Human checkpoints. Agents can "pause for human feedback at checkpoints or when encountering blockers" — a deliberate stop that hands control back to a person before a high-stakes or irreversible action.

Error handling lives in how observations are fed back. Because the agent gains "ground truth from the environment at each step," a tool error, a failed assertion, or an empty result is itself a useful observation: a well-built loop returns the error to the model so it can adapt on the next turn, rather than crashing. Reflection patterns generalize this — they turn a whole failed trajectory into feedback. The failure modes to design against are silent loops (the agent repeats an ineffective action because nothing tells it the action failed), error propagation (an early hallucinated fact corrupts every later step — exactly what grounding via tools mitigates), and runaway cost (no budget on a loop that never converges).

When a loop beats a fixed chain

A dynamic loop is more capable but also less predictable and more expensive than a straight-line workflow, so it should be earned, not assumed. Anthropic's guidance is to "find the simplest solution possible, and only increase complexity when needed" — for many tasks, "optimizing single LLM calls with retrieval and in-context examples is usually enough," and agentic systems should be reached for only when "simpler solutions fall short."

A useful rule of thumb: prefer a fixed chain or workflow when the steps are known in advance and the same every time (the path is predictable, so hard-code it — see orchestration-patterns.html for prompt chaining, routing, and the orchestrator-workers and evaluator-optimizer patterns). Prefer an agentic loop when the number and order of steps depend on intermediate results that can't be known ahead of time — open-ended research, debugging, or any task where the model must "dynamically direct" its own path. The cost of the loop (more tokens, more latency, harder-to-predict behavior) is the price of that flexibility, which is exactly why stopping conditions and ground-truth feedback are not optional extras but the things that make a loop safe to run.

The mechanism that makes any of this possible — the typed, named action space the loop draws from — is the subject of foundations-tool-use.html, and the context an agent carries across turns is covered in foundations-memory.html.