00 · Context
Splitting work by skill is just the tip of the iceberg.
When you actually build a Multi-Agent system, the question of "who does what" is only the first layer. The deeper layer is how the agents coordinate with each other — and that's where orchestration design patterns come in.
Each pattern isn't a "type of system" but a tactic for solving one specific class of problem. Hierarchical solves audit + scope. Routing solves cost-quality matching. Debate solves irreversible decisions. Each is an answer to a different question.
In production, you rarely use just one pattern. A real deep research agent is Orchestrator-Workers on the outer layer + Parallelization for fetch + Evaluator-Optimizer for synthesis. Recognizing which pattern sits at which layer of the stack matters more than labeling the whole system.
Patterns aren't an "either/or" choice — they're composable. You pick one pattern per design axis, not one pattern for the whole system. This piece walks through 7 patterns in increasing order of complexity, each with an SME-scale use case to make it concrete, and closes with a 6-level decision framework for how an orchestrator picks the right pattern.
01 · The pattern set
Seven models, seven tactics.
The first five are the canonical set most commonly seen in today's agentic AI literature. The last two — Parallelization and Orchestrator-Workers — belong to Anthropic's official set but are often left off the list.
A
Structural
Hierarchical / Supervisor
A "manager" agent coordinates sub-agents in a hierarchy tree. The manager doesn't do the work itself — it assigns and aggregates. Static org structure, vertical communication: assignments go down, reports come up.
SME Use Case
Multi-channel online store — a Supervisor takes messages from 3 channels and sorts them into 4 domains: Order Tracker, Product Q&A, Returns Agent, Escalation. Saves 2 support roles ≈ 15M VND/month.
Use when
Clear org chart, need an audit trail, scope is fixed hard
Avoid when
Exploratory tasks, shifting domain, need horizontal collaboration
B
Structural
Dynamic Routing
A router agent classifies intent and routes to the right specialist. The router is small & cheap — it only reads the input and classifies. Each specialist uses a different model/tool, optimizing cost per task type.
SME Use Case
Dental clinic — the Router classifies: Booking (cheap model), Pricing (RAG), Triage (medical model), Complaint (escalate). Cost ~500K–1M VND/month for 1,500 inquiries.
Use when
Cost/latency matter, heterogeneous task types, high volume
Avoid when
Tasks need many steps, domain too narrow, a wrong route breaks everything
C
Decomposition
Sequential Chain
The output of one agent is the input of the next, in a fixed order. Easiest to debug but also most brittle, because errors accumulate down the chain. Each step is an agent with reasoning, not a pure function.
SME Use Case
Marketing agency — 6 steps: Brief Parser → Topic Researcher → Outline Writer → Draft Writer → SEO Optimizer → Brand Voice QA. Boosts throughput 3–4x.
Use when
Standardized process, clear I/O schema, need high determinism
Avoid when
Tasks branch, need exploration, steps could run in parallel
D
Quality Control
Consensus / Debate
Several agents argue opposing views, and one judge agent renders the verdict. Bull finds reasons to do it; Bear finds the risks; Neutral brings objective data. Token cost is 3–5x a single pass.
SME Use Case
An F&B chain weighing a new location — Bull Agent, Bear Agent, Comparable Researcher and a Synthesis Agent that pulls it into a memo. The cost of getting it wrong = a full year's profit.
Use when
Irreversible decision, high cost-of-error, subjective
Avoid when
Ground truth exists, high volume, latency-sensitive
E
Quality Control
Evaluator-Optimizer
The Generator produces output, the Critic pushes back, the Generator revises. Loop until it hits the threshold. Exploits an asymmetry: judging is easier than generating. The Critic can be a smaller LLM or a test runner.
SME Use Case
Cross-border store — the Generator writes descriptions, the Critic scores them against a checklist (keyword density, title length, CTA). Loop 2–3 times. Cost 5–10K VND/product vs 100–200K for a freelancer.
Use when
Clear rubric, hard to one-shot, cheap Critic, large volume
Avoid when
Subjective criteria, Generator can one-shot it, latency-critical
F
Decomposition
Parallelization
Split a task into independent parts, run them in parallel, then merge the results. Two variants: Sectioning (split differently, optimize throughput) and Voting (repeat the same, reduce variance).
SME Use Case
Apartment rental company — Sectioning per application: ID Verifier, Income Checker, Reference Drafter, Risk Flagger, Aggregator. 33 hours/month → 2–3 hours of review.
Use when
Independent subtasks, need low latency, resources OK
Avoid when
Subtasks have dependencies, token cost multiplies significantly
G
Dynamic Decomposition
Orchestrator-Workers
Unlike Hierarchical: the structure is dynamic. The Orchestrator decomposes the task at runtime — it doesn't know in advance how many workers it needs or what they'll do. The backbone of deep research products.
SME Use Case
B2B sales prospect research — spawn dynamically: find basic info → detect funding → analyze founder's LinkedIn → find POS gaps. 30 min → 3–5 min, 4 calls/day → 12–15.
Use when
Scope unknown up front, exploratory task, every case differs
Avoid when
Standardized task, predictable latency, tight token budget
02 · Decision framework
How does an orchestrator know when to use which pattern?
The honest answer: mostly it doesn't really "know" — it's programmed ahead of time. But that programming comes in 6 levels, from crude to nearly autonomous.
01
Rule-based
Hard-coded conditional logic
Hard if/else. "If the task contains 'research' → Orchestrator-Workers." Predictable & auditable but extremely brittle, failing silently on new tasks.
02
Semantic Routing
A small LLM as the Router
A cheap LLM reads the task and classifies it into one of N patterns. Understands paraphrasing & hidden intent. Still bounded by a human-defined taxonomy.
03
Meta-reasoning
The LLM reasons about how to proceed
An orchestrator LLM analyzes which approach fits the task. The most flexible option, and it handles novel tasks — but the LLM gets overconfident when self-assessing.
04
Two-Phase Analysis
Task profiling → Pattern selection
Phase 1: extract quantitative features. Phase 2: map features → pattern via a decision tree. The decision is separated from the input, so it's easy to debug.
05
Escalation Policy
Start cheap — escalate when needed
Cheap ReAct first → Plan-and-Execute → Orchestrator-Workers → human-in-the-loop. Robust in practice, with no need for perfect prediction.
06
Self-correcting
Adaptive pattern switching · Frontier
The running pattern detects on its own that it's a poor fit and triggers a switch. Confidence drops, loop detection and budget exhaustion are the signals. Not yet stable in production.
03 · Composability
Patterns aren't mutually exclusive — they layer.
In a real system, you pick one pattern per design axis. There's no such thing as "my system is Hierarchical" — it's "topology Hierarchical + communication Direct + decomposition Sequential + quality control Evaluator".
Axis · 01
Topology
Hierarchical · Swarm · Recursive · Flat
Axis · 02
Communication
Direct call · Blackboard · Event-driven · Stigmergy
Axis · 03
Decomposition
Sequential · Parallel · Orchestrator-Workers · Plan-and-Execute
Axis · 04
Quality Control
Evaluator-Optimizer · Debate · Voting · None
Axis · 05
Routing
Dynamic Routing · MoE · Market-based · Static
Axis · 06
Reliability
Saga · Human-in-the-Loop · Speculative · Retry
"Most agents are over-engineered. Start with one well-designed pattern. Add complexity only when measurement shows it's needed."
Anthropic · Building Effective Agents