Workflow or agent? Climb one rung at a time

Suppose you run an online homeware shop, "Ví Dụ Home", with three customer-care staff and about 250 messages a day. A vendor pitches: "an AI agent that answers every message, live in four weeks". The right question is not "should we buy an agent" but which jobs need a model that chooses its own steps and which need only one call or a fixed assembly line. This post uses two sources to set the order of attempts: Anthropic's piece on building effective agents, and Yao and colleagues' ReAct paper. See also Workflows versus agents, Five LLM workflow patterns and The reason–act–observe loop.

How do workflows and agents differ, and why try the simple one first?

A workflow follows code paths fixed in advance; an agent steers itself. In Anthropic's terms, workflows are systems where models and tools are orchestrated through predefined code paths, while agents are systems where the model directs its own process and tool use (Anthropic, 2024). The company calls both agentic systems.

The core advice is to start simple. Anthropic recommends the simplest solution possible, adding complexity only when needed, which may mean building no agentic system at all; for many applications a single model call with retrieval and examples is usually enough (Anthropic, 2024). See the glossary entries on workflows and AI agents.

Where do the five workflow patterns sit?

I sort them into four rungs. That is my scheme, not Anthropic's, which files all five patterns under workflows. Rung 1 is one model call with retrieval and examples. Rung 2 is a fixed workflow, covering four of the five patterns. Rung 3 is orchestrator-workers, where subtasks are not predefined. Rung 4 is the agent. The "fits when" column below is from Anthropic.

The parallelisation pattern's own illustration is a guardrail: one model instance handles the user's query while another screens it, which Anthropic says tends to perform better than having one call do both (Anthropic, 2024). That connects to the post on the dealer chatbot that "agreed" to sell a car for $1.

Which five questions decide the rung?

I have gathered five questions. Each leans on something Anthropic says, but the order and grouping are mine.

1. Is one call with documents and examples already enough? If so, stop at rung 1 (Anthropic, 2024).
2. Does the job split cleanly into steps known in advance? If so, use a prompt chain: each step is easier, at the price of speed (Anthropic, 2024).
3. Can the inputs be sorted into a few clear groups handled separately? If so, add routing (Anthropic, 2024).
4. Are there clear criteria one call can use to grade another? If so, add an evaluator-optimiser loop (Anthropic, 2024).
5. Is the number of steps genuinely unpredictable, and do you trust the model's decisions in this setting? Only if both hold, consider an agent (Anthropic, 2024).

To be fair, Anthropic names customer support as a natural fit for more open-ended agents: conversations come with tools that pull order data, actions like refunds can be handled programmatically, and success is measurable through resolutions (Anthropic, 2024). The appendix adds that agents add the most value where there are clear success criteria, feedback loops and meaningful human oversight (Anthropic, 2024).

What evidence shows where the agent loop breaks?

The ReAct paper is the rare piece of evidence I read in full, but it shows kinds of failure, not that agents beat workflows in business. Yao and colleagues prompted the PaLM-540B model to interleave thoughts and actions (search, lookup, finish) over Wikipedia and compared it with plain chain-of-thought (CoT) reasoning (Yao et al., 2023). The results were mixed: on HotpotQA, ReAct scored 27.4 exact match against CoT's 29.4; on Fever, 60.9% against 56.3% (Yao et al., 2023).

The error analysis, on HotpotQA, is the part worth reading. From 50 right and 50 wrong trajectories sampled for each method, the authors found that 56% of CoT's failures were hallucinated facts against 0% for ReAct, which stays grounded in what its searches return. The price: 47% of ReAct's failures were reasoning errors against 16% for CoT, and 23% were unhelpful search results that derailed the model; about 29% were right answers that did not match the label (Yao et al., 2023). One failure pattern is specific to ReAct: it repeats its previous thought and action and cannot jump out of the loop (Yao et al., 2023). Anthropic also mentions stopping conditions such as a maximum number of iterations (Anthropic, 2024); my reading is that the looping failure shows why they matter.

The authors capped runs at 7 steps for HotpotQA and 5 for Fever, because more steps did not help: among trajectories with correct answers, those using all 7 or 5 steps made up only 0.84% and 1.33%. When ReAct ran out of steps without answering, they backed off to CoT with self-consistency voting (CoT-SC), and that combination scored highest on HotpotQA (35.1) among the prompting methods (Yao et al., 2023). The operating lesson: an agent needs a step ceiling and a fallback to something simpler. The source's limit is that its tasks were question answering, fact checking and a text game from 2022, not business processes.

Which rung should each job at a hypothetical shop take?

Every number here is hypothetical. Ví Dụ Home gets about 250 messages a day. I split them by group and assign a rung from the five questions above, with an assumed number of model calls per case.

On this table 85% of the volume runs on one or two calls, 5% on three, and only 0.5% of cases need an agent. Handing large refunds to a person follows the intervention trigger OpenAI describes: high-risk actions such as authorising large refunds or making payments (OpenAI, 2025). Now the arithmetic, assuming a five-day week: 0.5% of 250 messages a day is about 6 lost-parcel cases a week at 15 minutes each, which is 1.5 hours a week. At that number an agent is probably not worth building. If the share rose to 5%, about 60 cases a week, the sum would change.

How do you know it is time to move up a rung?

Run the lower rung first on about 50 past cases with known right answers. This test is mine, not the sources'; it leans on Anthropic's point that complexity should be added only when it demonstrably improves outcomes (Anthropic, 2024). Fix a threshold in advance, for example at least 90% correct. If the lower rung passes, stop. If not, look at the kind of failure: missing information (add retrieval), wrongly split steps (change the pattern), or a genuinely unpredictable number of steps (move up), then rerun the same 50 cases. To fill in the five cells for an agent you have chosen, see the five ingredients of an AI agent.

Going deeper: what I read and what I did not

I read Anthropic (2024) in full; it was published on 19 December 2024 and the page notes that much of the tooling it describes has changed since, so the copy I read was edited after publication and I use it only for concepts. It is written by an AI company for developers, not independent research. I read the arXiv text of Yao and colleagues (2023) in full, but not the appendix trajectory by trajectory. I use OpenAI (2025) for one point only (when to hand over to a person), from a text extraction of the PDF.

The four rungs, five questions and 50-case test are mine. The shop Ví Dụ Home, the shares of message types, calls per case, the 500,000 dong limit and the 90% threshold are numbers I made up to illustrate; I have not measured any of this on a real process.