---
title: "Workflow or AI Agent: Which Rung for Your Process"
description: "Four rungs from one AI call to an agent, five questions to choose a rung, and where agent loops fail according to the ReAct paper. Hypothetical shop."
url: "https://staging.khoi.co.uk/en/blog/workflow-hay-agent-leo-tung-bac"
lang: en-GB
author: "Lê Quốc Khôi"
date: 2026-11-03
updated: 2026-11-03
topics: ["AI for leaders"]
tags: ["AI workflow", "AI agents", "ReAct", "Process automation", "AI for SMEs"]
---

# Workflow or agent? Climb one rung at a time

## Key takeaways

- Anthropic defines workflows as systems where models and tools are orchestrated through predefined code paths, and agents as systems where the model directs its own process; it advises finding the simplest solution, and says that for many applications a single model call with retrieval and examples is enough (Anthropic, 2024).
- Anthropic's five workflow patterns are prompt chaining, routing, parallelisation, orchestrator-workers and evaluator-optimiser (Anthropic, 2024). Arranging them into four rungs (single call, fixed workflow, orchestrator-workers, agent) is my own scheme.
- ReAct (Yao and colleagues, 2023): among sampled wrong answers on research-style question answering, none of ReAct's came from fabricated facts (CoT: 56%), but it had more reasoning errors, including repeating the same thought and action; the authors capped it at 7 and 5 steps (Yao et al., 2023).
- The five rung-choosing questions and the 50-case test are mine, not the sources'.
- The shop "Ví Dụ Home" and every number in it are hypothetical, invented by me.

Suppose you run an online homeware shop, "Ví Dụ Home", with three customer-care staff and about 250 messages a day. A vendor pitches: "an AI agent that answers every message, live in four weeks". The right question is not "should we buy an agent" but which jobs need a model that chooses its own steps and which need only one call or a fixed assembly line. This post uses two sources to set the order of attempts: Anthropic's piece on building effective agents, and Yao and colleagues' ReAct paper. See also [Workflows versus agents](https://staging.khoi.co.uk/en/framework/workflows-vs-agents), [Five LLM workflow patterns](https://staging.khoi.co.uk/en/framework/agentic-workflow-patterns) and [The reason–act–observe loop](https://staging.khoi.co.uk/en/framework/react-agent-loop).

## How do workflows and agents differ, and why try the simple one first?

A workflow follows code paths fixed in advance; an agent steers itself. In Anthropic's terms, workflows are systems where models and tools are orchestrated through predefined code paths, while agents are systems where the model directs its own process and tool use (Anthropic, 2024). The company calls both agentic systems.

The core advice is to start simple. Anthropic recommends the simplest solution possible, adding complexity only when needed, which may mean building no agentic system at all; for many applications a single model call with retrieval and examples is usually enough (Anthropic, 2024). See the glossary entries on [workflows](https://staging.khoi.co.uk/en/glossary#llm-workflow) and [AI agents](https://staging.khoi.co.uk/en/glossary#ai-agent).

> **Figure:** [Workflows versus agents](https://staging.khoi.co.uk/en/framework/workflows-vs-agents) Workflow versus agent in Anthropic's distinction (2024); the comparison table is the site's own arrangement.

## Where do the five workflow patterns sit?

I sort them into four rungs. That is my scheme, not Anthropic's, which files all five patterns under workflows. Rung 1 is one model call with retrieval and examples. Rung 2 is a fixed workflow, covering four of the five patterns. Rung 3 is orchestrator-workers, where subtasks are not predefined. Rung 4 is the agent. The "fits when" column below is from Anthropic.

| Pattern (rung) | What it does | Fits when (per Anthropic) |
|:--|:--|:--|
| Single call + retrieval (1) | One call, with documents and examples | Enough for many applications |
| Prompt chaining (2) | Sequential steps; each call processes the previous output | Task splits cleanly into fixed steps; trades latency for accuracy |
| Routing (2) | Classify the input, send it to a specialised process | Distinct categories, accurately classified |
| Parallelisation (2) | Run independent subtasks at once, or repeat a task and vote | Speed, or several perspectives for confidence |
| Evaluator-optimiser (2) | One call drafts, another evaluates and gives feedback, in a loop | Clear evaluation criteria; iteration adds measurable value |
| Orchestrator-workers (3) | A central model splits the task, delegates, merges results | Complex tasks where subtasks cannot be predicted |
| Agent (4) | The model plans and uses tools in a loop | Open-ended problems, unknown number of steps, trust in the model's decisions |

The parallelisation pattern's own illustration is a guardrail: one model instance handles the user's query while another screens it, which Anthropic says tends to perform better than having one call do both (Anthropic, 2024). That connects to the post on the [dealer chatbot that "agreed" to sell a car for $1](https://staging.khoi.co.uk/en/blog/chatbot-xe-mot-do-rao-chan).

## Which five questions decide the rung?

I have gathered five questions. Each leans on something Anthropic says, but the order and grouping are mine.

1. Is one call with documents and examples already enough? If so, stop at rung 1 (Anthropic, 2024).
2. Does the job split cleanly into steps known in advance? If so, use a prompt chain: each step is easier, at the price of speed (Anthropic, 2024).
3. Can the inputs be sorted into a few clear groups handled separately? If so, add routing (Anthropic, 2024).
4. Are there clear criteria one call can use to grade another? If so, add an evaluator-optimiser loop (Anthropic, 2024).
5. Is the number of steps genuinely unpredictable, and do you trust the model's decisions in this setting? Only if both hold, consider an agent (Anthropic, 2024).

To be fair, Anthropic names customer support as a natural fit for more open-ended agents: conversations come with tools that pull order data, actions like refunds can be handled programmatically, and success is measurable through resolutions (Anthropic, 2024). The appendix adds that agents add the most value where there are clear success criteria, feedback loops and meaningful human oversight (Anthropic, 2024).

## What evidence shows where the agent loop breaks?

The ReAct paper is the rare piece of evidence I read in full, but it shows kinds of failure, not that agents beat workflows in business. Yao and colleagues prompted the PaLM-540B model to interleave thoughts and actions (search, lookup, finish) over Wikipedia and compared it with plain chain-of-thought (CoT) reasoning (Yao et al., 2023). The results were mixed: on HotpotQA, ReAct scored 27.4 exact match against CoT's 29.4; on Fever, 60.9% against 56.3% (Yao et al., 2023).

The error analysis, on HotpotQA, is the part worth reading. From 50 right and 50 wrong trajectories sampled for each method, the authors found that 56% of CoT's failures were hallucinated facts against 0% for ReAct, which stays grounded in what its searches return. The price: 47% of ReAct's failures were reasoning errors against 16% for CoT, and 23% were unhelpful search results that derailed the model; about 29% were right answers that did not match the label (Yao et al., 2023). One failure pattern is specific to ReAct: it repeats its previous thought and action and cannot jump out of the loop (Yao et al., 2023). Anthropic also mentions stopping conditions such as a maximum number of iterations (Anthropic, 2024); my reading is that the looping failure shows why they matter.

The authors capped runs at 7 steps for HotpotQA and 5 for Fever, because more steps did not help: among trajectories with correct answers, those using all 7 or 5 steps made up only 0.84% and 1.33%. When ReAct ran out of steps without answering, they backed off to CoT with self-consistency voting (CoT-SC), and that combination scored highest on HotpotQA (35.1) among the prompting methods (Yao et al., 2023). The operating lesson: an agent needs a step ceiling and a fallback to something simpler. The source's limit is that its tasks were question answering, fact checking and a text game from 2022, not business processes.

## Which rung should each job at a hypothetical shop take?

Every number here is hypothetical. Ví Dụ Home gets about 250 messages a day. I split them by group and assign a rung from the five questions above, with an assumed number of model calls per case.

| Job | Assumed share | Rung | Calls per case | Reason |
|:--|:-:|:-:|:-:|:--|
| Order status | 45% | 2 (routing) | 2 | Code looks up the order; the model only classifies and words the reply |
| Product questions | 25% | 1 | 1 | The answer sits on the product page |
| Returns, complaints | 15% | 2 (routing + person) | 2 | Refunds above 500,000 dong go to a person to approve |
| Replies to bad reviews | 5% | 2 (chain) | 3 | Steps known in advance; the draft goes to a person to post |
| Lost-parcel investigation | 0.5% | 4 (agent) | about 8 | Unpredictable steps: check order, ask the carrier, read the warehouse slip |
| Other | 9.5% | hand to a person | 0 | Not yet worth automating |

On this table 85% of the volume runs on one or two calls, 5% on three, and only 0.5% of cases need an agent. Handing large refunds to a person follows the intervention trigger OpenAI describes: high-risk actions such as authorising large refunds or making payments (OpenAI, 2025). Now the arithmetic, assuming a five-day week: 0.5% of 250 messages a day is about 6 lost-parcel cases a week at 15 minutes each, which is 1.5 hours a week. At that number an agent is probably not worth building. If the share rose to 5%, about 60 cases a week, the sum would change.

## How do you know it is time to move up a rung?

Run the lower rung first on about 50 past cases with known right answers. This test is mine, not the sources'; it leans on Anthropic's point that complexity should be added only when it demonstrably improves outcomes (Anthropic, 2024). Fix a threshold in advance, for example at least 90% correct. If the lower rung passes, stop. If not, look at the kind of failure: missing information (add retrieval), wrongly split steps (change the pattern), or a genuinely unpredictable number of steps (move up), then rerun the same 50 cases. To fill in the five cells for an agent you have chosen, see [the five ingredients of an AI agent](https://staging.khoi.co.uk/en/blog/nam-nguyen-lieu-cua-ai-agent).

## Going deeper: what I read and what I did not

I read Anthropic (2024) in full; it was published on 19 December 2024 and the page notes that much of the tooling it describes has changed since, so the copy I read was edited after publication and I use it only for concepts. It is written by an AI company for developers, not independent research. I read the arXiv text of Yao and colleagues (2023) in full, but not the appendix trajectory by trajectory. I use OpenAI (2025) for one point only (when to hand over to a person), from a text extraction of the PDF.

The four rungs, five questions and 50-case test are mine. The shop Ví Dụ Home, the shares of message types, calls per case, the 500,000 dong limit and the 90% threshold are numbers I made up to illustrate; I have not measured any of this on a real process.

## Frequently asked questions

### Is an agent always more expensive than a workflow?

Anthropic writes that agentic systems often trade latency and cost for better task performance, and that agents mean higher costs and a risk of compounding errors (Anthropic, 2024). It gives no figures, so the gap in your own process can only be found by measuring it.

### Is orchestrator-workers a workflow or an agent?

Anthropic files it under workflows: a central model splits the task, hands pieces to other models and combines the results; unlike parallelisation, the subtasks are not predefined but chosen by the orchestrator for each input (Anthropic, 2024). I put it on the rung closest to an agent, but the rung scheme is mine.

### A vendor says its product is an 'agent': what should I ask?

Ask which steps the code decides, which the model decides, and where the agent stops. Anthropic says an agent needs real results from its environment at each step, can pause for a person at checkpoints, and it is common to include stopping conditions such as a maximum number of iterations (Anthropic, 2024). A product called an agent that cannot answer those three is hard to judge.

### Does the ReAct paper, first posted in 2022, still hold for today's models?

I do not know. The main experiments used the PaLM-540B model with prompting, on question answering, fact checking and a text game, with a Wikipedia lookup tool for the first two (Yao et al., 2023). I use it to show the kinds of failure to watch for, such as looping, not to predict how accurate current models are.

## References

- Anthropic (2024) *Building effective AI agents*. Anthropic Research. Available at: https://www.anthropic.com/research/building-effective-agents (Accessed: 2026-11-01).
- OpenAI (2025) *A practical guide to building agents [guide]*. Available at: https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf (Accessed: 2 November 2026).
- Yao, S. et al. (2023) *ReAct: synergizing reasoning and acting in language models*. (ICLR 2023 (camera-ready version); arXiv:2210.03629). https://doi.org/10.48550/arXiv.2210.03629

---

Canonical HTML version: https://staging.khoi.co.uk/en/blog/workflow-hay-agent-leo-tung-bac