What Are AI Agents? Workflows, ReAct and Real Success Rates (2026)

Most of the "agents" I get asked to review turn out to be a prompt, a for loop and a hope. That is not an insult: the for loop is the agent. I run PandaStack, a Firecracker microVM cloud that agents use to execute code (so I see the runtime end of a lot of agent systems), and the question I hear most often from infrastructure teams is still the basic one: what are AI agents, really, and when is one worth building? This post answers that for engineers. It covers the workflows versus agents distinction from Anthropic's guidance, the ReAct loop and tool calling, what planning and memory actually mean in code, when multi-agent designs pay off, the success rates measured on public benchmarks in 2025 and 2026, and the cases where you should not build an agent at all.

What is an AI agent, in a definition you can test?

The definition I use is the one from Anthropic's Building Effective Agents, published on 19 December 2024 and still the clearest framing around. It splits "agentic systems" into two kinds. Workflows are systems where language models and tools are orchestrated through predefined code paths: your code decides the sequence. Agents are systems where the model dynamically directs its own process and tool usage: the model decides what happens next.

That gives you a test you can apply to any architecture diagram. Find the line of code that decides the next step. If it is an if statement or a DAG edge you wrote, you have a workflow. If it is the model's output, parsed and executed, you have an agent. Most production systems are hybrids: an agent loop inside one box of a workflow.

The marketing definition is looser: Gartner, in coverage of its June 2025 forecast, estimated that only about 130 of the thousands of vendors selling "agentic AI" were the real thing and called the rest "agent washing" (Outlook Business). A chatbot with a retrieval step is not an agent under either definition. It is a single model call with extra context.

Workflows vs agents: the five patterns worth knowing

Anthropic's post lists five workflow patterns before it gets to agents, and I find they cover most of what teams actually need:

Pattern	Who decides the next step	Good fit	Source
Prompt chaining	Your code, fixed sequence	Generate, check, then translate or format	Anthropic
Routing	A classifier call, then your code	Support triage, picking a cheap or expensive model	Anthropic
Parallelization	Your code fans out, then votes or merges	Guardrail checks, multiple reviewers on one diff	Anthropic
Orchestrator-workers	A lead model splits the task at run time	Research, multi-file code changes	Anthropic
Evaluator-optimizer	Loop until a judge model accepts	Translation, writing to a rubric	Anthropic
Autonomous agent	The model, every turn, until it stops	Open-ended tasks with an unknown number of steps	Anthropic

The post's advice is to start with the simplest option and add complexity only when it measurably helps, and to begin with the LLM API directly rather than a framework, because framework layers hide the prompts and responses you need to debug. I agree with both. If you do pick a framework, read its loop first; my comparison of the options is in AI agent frameworks compared.

How does an AI agent work? ReAct, tool calls and the loop

The pattern underneath almost every agent is ReAct, from the 2022 paper ReAct: Synergizing Reasoning and Acting in Language Models by Yao and colleagues (ICLR 2023). The idea is to interleave reasoning with actions: think, call a tool, observe the result, think again. With one or two in-context examples it beat imitation and reinforcement learning baselines by 34 and 10 absolute points of success rate on ALFWorld and WebShop.

In 2022 that meant parsing "Action:" lines out of free text. Today the model APIs do it natively. You send tool definitions (a name, a description, a JSON Schema for the arguments), the model replies with a structured tool call, your code runs it, and you send the result back. On the Anthropic Messages API the response carries a stop_reason of tool_use when the model wants a tool and end_turn when it is done, and tool results go back as tool_result blocks that must come first in the next user message (handle tool calls). The whole agent fits on a screen:

# Minimal agent loop. pip install anthropic==1.15.0 (Python >= 3.10)
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY
TOOLS = [{
    "name": "get_disk_usage",
    "description": "Return disk usage percent for a mount point on this host.",
    "input_schema": {"type": "object",
                     "properties": {"mount": {"type": "string"}},
                     "required": ["mount"]},
}]

def get_disk_usage(mount: str) -> str:
    import shutil
    u = shutil.disk_usage(mount)
    return f"{100 * u.used / u.total:.1f}% used"

messages = [{"role": "user", "content": "Is / more than 80% full?"}]
for _ in range(10):                      # hard cap: never loop forever
    resp = client.messages.create(model="claude-sonnet-5-5", max_tokens=4096,
                                  tools=TOOLS, messages=messages)
    messages.append({"role": "assistant", "content": resp.content})
    if resp.stop_reason != "tool_use":   # end_turn, refusal, max_tokens...
        break
    results = [{"type": "tool_result", "tool_use_id": b.id,
                "content": get_disk_usage(**b.input)}
               for b in resp.content if b.type == "tool_use"]
    messages.append({"role": "user", "content": results})

print(next((b.text for b in resp.content if b.type == "text"), ""))

Everything else is controls around this loop: retries, logging, budgets, approvals, sandboxes. The hands-on version, with error handling, structured logs, a token budget and a second provider, is in the sibling guide, How to Build an AI Agent With Tool Calling in Python.

Two current details catch people. Claude Opus 5.5 and Sonnet 5.5 reject forced tool use (tool_choice of any or a named tool returns a 400), so the docs point you to auto plus strict schemas instead (define tools). And thinking is on by default on the 5.x models, which is one more reason to append the assistant's content back unchanged rather than rebuilding it.

Planning and memory: what the parts actually are

"Planning" in diagrams sounds like a module. In practice it is one of three things. The model plans implicitly inside its reasoning before each tool call (the ReAct default). Or you ask it to write an explicit plan, often as a todo list stored in a file or a tool, and re-read it each turn so it survives long contexts. Or an orchestrator model writes the plan and hands pieces to workers. Anthropic's second core principle, transparency, is really a request to make the plan visible so a human can see where the agent thinks it is.

"Memory" is similarly plain. Short-term memory is the message list you keep appending to. It is bounded by the context window (1M tokens on the current Claude models per the models overview), and long before you hit that the quality drops and the bill grows, because every turn resends the whole history. Long-term memory is anything you store outside the context and retrieve later: files, a database, a vector index, a summary written at the end of a session. My survey of memory products found a plain filesystem competitive with vendor systems, which matches my bias: start with files and grep, and add retrieval infrastructure when you can show it helps on your evals.

Single agent or multi-agent?

The two best primary sources on this disagree, usefully.

Anthropic's write-up of its multi-agent research system (13 June 2025) reports that a lead agent on Claude Opus 4 with Sonnet 4 subagents beat single-agent Opus 4 by 90.2% on its internal research eval. The cost is the headline you should remember: agents use about 4 times the tokens of a chat, and multi-agent systems about 15 times. On BrowseComp, token usage alone explained 80% of the performance variance. The same post says multi-agent designs fit poorly when agents need shared context or have many dependencies, and that most coding tasks have fewer truly parallel parts than research.

A day earlier Cognition's Walden Yan published Don't Build Multi-Agents (12 June 2025), arguing from two principles: share full context and agent traces, not just messages, and remember that actions carry implicit decisions, so parallel subagents make conflicting ones. His recommendation is a single-threaded agent, with history compression for long tasks.

Read together, they say the same thing. Fan out when the subtasks are genuinely independent and reading-heavy (search, research, evaluation), and keep one thread when the subtasks write to shared state (code, infrastructure, anything with side effects). Then check whether the improvement is worth roughly four times the tokens of a single agent.

How often do AI agents actually succeed?

Less often than demos suggest, and the number depends entirely on the task. These are the measurements I trust enough to quote:

Measurement	Result	Source
OSWorld (real desktop tasks), best agent	About 5% in 2024, 66.3% in 2025, human baseline 72.35%	Stanford AI Index 2026
TheAgentCompany (simulated software firm), best agent	30% of tasks completed autonomously	arXiv 2412.14161 v3
tau-bench retail, GPT-4o	Under 50% of tasks; pass^8 below 25%	arXiv 2406.12045
METR time horizon, GPT-5 agent	About 2 hours 17 minutes of expert task time at 50% reliability	METR
Organisations experimenting with agents / scaling them	62% / 23%, and no more than 10% scaling in any single function	McKinsey State of AI 2025, IT Brief
Agentic AI projects cancelled by end of 2027 (forecast)	Over 40%	Gartner via Outlook Business

Three patterns come out of this. First, structured computer tasks are close to human level now, but open-ended professional work is not: 30% on a simulated company is a long way from 66% on a desktop benchmark. Second, reliability decays with length. METR's framing is the useful one: an agent's success rate falls as the human time for the task grows, so short tasks are a good bet and day-long ones are not. Third, consistency is its own problem. tau-bench's pass^k metric asks whether the agent succeeds on all k tries of the same task, and the drop from one try to eight is what your on-call rotation will feel. A demo shows pass@1. Production needs something closer to pass^k. I wrote up the specific ways this goes wrong in AI agents in production: failure modes.

When should you not build an AI agent?

Building Effective Agents says agents trade latency and cost for better task performance. That trade is often a bad one, and I would not build an agent when:

• The steps are known. If you can draw the flowchart, write it as a workflow. It will be cheaper, faster, testable with ordinary unit tests, and it will fail in ways you can predict.
• One call is enough. A single model call with good retrieval and a few examples solves a surprising share of "agent" requests. Anthropic's post makes this point first.
• Errors are expensive and irreversible. An agent at 66% or even 90% task success needs a human approval step before anything that deletes, pays or deploys. If you cannot afford that step, the task is not ready for an agent.
• The economics do not work. Each turn resends the whole conversation. On Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens (pricing), a 10-turn run averaging 20,000 input tokens and 1,000 output tokens per turn costs about $0.40 for input and $0.10 for output, roughly $0.50 before prompt caching. That is cheap for a task that saves an engineer an hour and ruinous for one that replaces a database lookup.

This is where platform discipline earns its keep again. An agent is a non-deterministic process with credentials, so it needs the boring things: least-privilege tool permissions, an isolated place to run code (see AI agent sandboxes explained), structured logs per tool call, budgets, timeouts and an off switch. None of that is new. All of it is skipped in most first agent projects.

FAQ

What is the difference between an AI agent and a chatbot?

A chatbot answers in one model call, possibly with retrieved context. An agent runs a loop in which the model chooses tools, sees their results and decides the next step until the task is done. The test is who decides the next step: your code (workflow or chatbot) or the model (agent).

What is the ReAct pattern in AI agents?

ReAct, from a 2022 paper by Yao et al., interleaves reasoning steps with actions such as tool calls and feeds each observation back to the model. Modern tool-calling APIs implement it natively: the model returns a structured tool call, your code executes it, and the result goes back as the next message.

Are multi-agent systems better than single agents?

Sometimes. Anthropic measured a 90.2% improvement on research tasks with a multi-agent design, at about 15 times the tokens of a chat. Cognition argues single-threaded agents are more reliable when subtasks share state. Use multiple agents for independent, read-heavy subtasks and a single agent for anything that writes.

How reliable are AI agents in 2026?

It depends on task length and structure. The best agents reached 66.3% on OSWorld desktop tasks (humans score 72.35%), but 30% on TheAgentCompany's realistic office tasks. Expect high success on short, well-scoped tasks and plan for human review on long or irreversible ones.

This quarter I would do three things if I were starting an agent project. I would write the task as a workflow first and only promote the step that genuinely needs model judgement into an agent loop. I would build an eval set of 30 to 50 real tasks and track pass^k, not a single lucky run. And I would put the loop behind the same controls as any other service with credentials: scoped tools, a sandbox, per-call logs and a hard budget. The model will keep getting better on its own. The controls will not.

Related: AI Agent Sandboxes: What They Are and How to Choose One, Agent Observability in 2026: The Spans Are Standard, the Standard Isn't Stable, and Nobody Reads the Logs and It's 2 AM. Do You Know What Your AI Agent Is Doing?.