Multi-agent AI: the complete design guide

Quick answer

A multi-agent AI system is a workflow where multiple AI agents with defined roles coordinate to complete a task that no single agent handles end to end. Each agent has one job, a set of tools, a model suited to its role, and a handoff point where its output becomes another agent's input. The design process goes: objective → roles → execution pattern → tool access → model assignment → human checkpoints — in that order, before any code runs.

What is a multi-agent AI system?

A multi-agent AI system is a workflow where multiple AI agents work together to complete a task that no single agent handles end to end. Each agent has a defined role, a set of tools it can use, a model suited to its job, and a clear handoff point where its output becomes another agent's input.

The core idea is division of labour. A single large language model can do many things, but it does all of them at once — without the kind of specialisation, scope control, and accountability structure that makes complex work reliable. Multi-agent systems bring those properties back by giving each agent a narrower job.

Think of it like a team. A solo consultant can cover strategy, delivery, and client management. But for complex engagements, you bring in specialists: a strategist, a delivery lead, a subject matter expert, a project manager. The work is divided because division produces better output, not because any one person couldn't theoretically attempt the whole thing.

Multi-agent AI works the same way. A customer support workflow might have a classifier that reads the ticket, a retrieval agent that fetches relevant documentation, a drafter that writes the response, and a reviewer that checks it before sending. Each step is narrower, more reliable, and easier to improve in isolation than a single agent trying to do all four things at once.

When to use multi-agent — and when not to

Multi-agent is not always the right choice. The architecture introduces real coordination overhead: agents need to pass context to each other, errors can propagate through the pipeline, and debugging becomes harder when a failure happens three steps into a five-step workflow.

Use a multi-agent system when:

• The task has genuinely distinct phases that require different capabilities (research vs. writing vs. review)
• Different steps need different models — a cheap fast model for classification, a capable model for drafting, a specialised model for legal or technical review
• You need human approval at specific checkpoints before the workflow continues
• Volume is high enough that parallelising independent work meaningfully reduces latency
• You need to isolate accountability — to know exactly which step produced an incorrect output
• The workflow needs to be auditable: every agent's input and output logged, reviewable, traceable

Stick with a single agent when:

• The task is short enough to complete in a single context window without degradation
• All steps require the same model and tool access
• The coordination cost (passing context, handling failures across steps) exceeds the benefit
• You're still figuring out what the workflow should be — single agents are faster to iterate

The most common mistake teams make is building multi-agent systems prematurely. They reach for orchestration frameworks before they understand the task. Start with the simplest thing that could work; introduce specialisation when the limitation is real, not theoretical.

The five design decisions

Every multi-agent system comes down to five design decisions. Get these right in the design phase and the implementation is straightforward. Skip them and you'll rebuild the system two or three times.

1. Agent roles

Each agent needs a clearly named role with a defined scope. A role is not a task description — it's an identity. "Research Agent" is a role. "Search for information" is a task. The distinction matters because a well-defined role tells the agent what it is and isn't responsible for, which shapes how it behaves at the edges of its scope.

A good role definition includes:

• A name that describes the function, not the action
• A one-sentence scope statement (what this agent is responsible for)
• What it should not do — the explicit out-of-scope boundaries
• The success condition: what "done" looks like for this agent

Closely tied to role is skills — the instructions, rules, and operating procedures the agent actually runs on. If the role is what the agent is, the skills are how it does its job. A Research Agent's skills might specify how to evaluate source quality, what to do when search results conflict, and how to format its output so the next agent can use it. Skills are declared per-agent in the Envelope schema and shape behaviour within the boundaries the role defines.

For a deeper treatment of what makes a role definition work, see What makes a good agent role definition. For the full five components — role, skills, model, tools, and handoffs — see Sub-agent design: the five components every agent needs.

2. Tools and access

Each agent should have access to exactly the tools it needs — no more. Tool access is not a configuration detail; it's a trust boundary. An agent that has access to tools it doesn't need has a larger blast radius when something goes wrong.

Map tool access per agent, not per workflow. A research agent might need web search and a knowledge base. A drafting agent needs only the research output and a text editor. A reviewer needs the draft and a rubric — no external API access required.

The Envelope schema encodes this as per-agent tools and accessPolicy declarations. The access policy specifies which outbound HTTP calls each agent is permitted to make — allowlisted by hostname, method, and path prefix. If the agent tries to call something not on the allowlist, the runtime blocks it.

This matters for security, compliance, and debugging. When an agent misbehaves, the first question is always "what did it have access to?" If the answer is "everything," the investigation is much harder.

3. Model routing

Different agents have different requirements. A classifier that sorts support tickets into ten categories needs speed and low cost — a small fast model is the right choice. A drafting agent producing legal summaries needs capability and precision — a frontier model is worth the cost. A code review agent might need a model with strong programming knowledge specifically.

Model routing is the practice of assigning each agent the model that's optimal for its job. This is distinct from picking a single model for the whole workflow. Multi-agent systems make model routing possible in a way single-agent systems don't: you can use the right tool for each step instead of making the same compromise everywhere.

The assignment should be explicit in the design — not left to the runtime or inferred from defaults. Every agent in an Envelope team definition has a declared model field. This makes the routing auditable and changeable without touching the agent's logic.

For a full breakdown of routing strategy, see Model routing in multi-agent workflows.

4. Handoffs and sequencing

How outputs move between agents is the architecture of the system. A handoff is a declaration that one agent's output becomes another agent's input, and that the downstream agent should not start until the upstream one is complete.

The two fundamental patterns are sequential and parallel:

Sequential: Agent B waits for Agent A. Use this when B genuinely needs A's output to do its job — a drafter waiting for a research brief, a reviewer waiting for a draft. In the Envelope schema, this is expressed with dependsOn.

Parallel: Agents A, B, and C run at the same time. Use this when their work is independent — three research agents covering different domains, two classifiers checking the same input against different rubrics. Parallel execution is the default in Envelope; you opt into sequencing, not out of parallelism.

For a practical decision framework with worked examples, see Parallel vs. sequential agents: when to use each.

The most common handoff mistake is passing too much context downstream. Agents should receive exactly what they need for their role. Flooding a downstream agent with the full history of the workflow's execution buries the relevant signal in noise and degrades output quality.

5. Human gates

A human gate is a checkpoint where a person must review and approve before the workflow continues. Gates are not a sign that the system isn't working — they're a design feature that keeps humans accountable for decisions with real consequences.

Gates should be designed in from the start, not added as an afterthought when something goes wrong. An afterthought gate is a patch on a workflow that was designed to run without oversight. A designed-in gate is a deliberate decision point: "at this moment in the workflow, a human must make a call."

Every gate needs a defined approve/reject/timeout behaviour. What happens if the reviewer approves? What happens if they reject — does the workflow rerun a step, or stop entirely? What happens if no one responds within 24 hours?

In the Envelope schema, gates are declared as humanGate steps with explicit onApprove, onReject, and timeout handlers. This makes the gate's behaviour part of the design, not an undocumented assumption.

For a full treatment of gate placement and specification, see Human-in-the-loop: how to design approval gates.

Core patterns

Multi-agent systems cluster into a handful of common patterns. Most real workflows combine several.

The pipeline

The simplest pattern: agents run in sequence, each one consuming the previous one's output. Research → Draft → Review → Send.

Best for: tasks with a natural linear flow, where each step genuinely transforms the output of the last.

Watch out for: error propagation — a poor output in step two affects everything downstream. Design recovery paths for each step.

The fan-out / fan-in

A coordinator agent distributes work to multiple specialist agents running in parallel, then aggregates their outputs.

Best for: tasks that can be decomposed into independent workstreams — covering multiple data sources, multiple domains, multiple languages at the same time.

Watch out for: aggregation logic is the hard part. What happens if two agents produce conflicting outputs? The aggregator needs a resolution strategy.

The routing pattern

A classifier agent reads the input and routes it to the right specialist. A support ticket goes to the billing agent, the technical agent, or the account management agent depending on what the classifier decides.

Best for: high-volume, heterogeneous inputs where different inputs require fundamentally different handling.

Watch out for: edge cases the classifier miscategorises. Build a fallback route for inputs that don't match any category cleanly.

The reflection loop

An agent produces output; a separate reviewer agent evaluates it against a rubric; if it fails, the original agent revises and the reviewer checks again. The loop continues until the reviewer is satisfied or a maximum iteration count is reached.

Best for: tasks where quality is hard to specify upfront but easy to evaluate — writing tasks, code generation, analysis with known quality criteria.

Watch out for: infinite loops. Always define a maximum iteration count and a "good enough" exit condition. A loop that runs six times produces diminishing returns and real cost.

The hierarchical team

A lead agent breaks a complex goal into subtasks, delegates each subtask to a specialist agent, collects the results, and synthesises a final output.

Best for: open-ended complex tasks where the decomposition itself is part of the work — research projects, strategic analysis, multi-part deliverables.

Watch out for: coordination overhead compounds at scale. A lead managing ten specialists is hard to debug when something goes wrong. Keep the hierarchy shallow.

Best practices

Design before you build

The most expensive multi-agent bugs are architectural. A handoff that was designed wrong, a gate that was never designed at all, a model routing decision that was never made explicit — these surface in production, not in unit tests.

Design the team on paper (or in a tool like Envelope) before writing any code. Define every role, every tool assignment, every handoff, every gate. Export a spec. Then build from the spec. The spec is not overhead — it's the design work that prevents rebuilds.

Name agents by function, not by technology

Call your agent "Account Research Specialist," not "GPT-4o Research Tool." Names should reflect the role in the business workflow, not the implementation. This matters for two reasons: it forces you to think clearly about what the agent is actually for, and it keeps the design legible to non-technical stakeholders who need to review and approve it.

Keep contexts lean

Every token you pass between agents has a cost — monetary and qualitative. Downstream agents that receive dense, noisy contexts produce worse outputs than agents that receive focused, relevant inputs. Design your handoffs to pass structured summaries, not raw transcripts of everything that happened upstream.

Build the unhappy paths

Most agents are designed for the happy path — the input is clean, the tools respond correctly, the output is usable. Real systems encounter the unhappy path constantly: ambiguous inputs, tool failures, empty search results, reviewer rejections.

Every agent that calls an external tool needs a failure handler. Every gate needs a timeout behaviour. Every agent whose output feeds a downstream step needs a validation check before the handoff. These are not edge cases; they are regular operating conditions.

Version your designs

When you change an agent's role, tool access, or model — even if the change seems small — the output characteristics change. A downstream agent that was trained (in the prompt-engineering sense) on the upstream agent's previous output format may break silently when that format changes.

Version your team definitions. Keep the old version running for live integrations while the new version goes through testing. Don't upgrade in place for production workflows.

Narrow by design

The temptation in multi-agent design is to give each agent broad scope and let it figure out what to do. Resist this. Narrow agents are more reliable, cheaper to run, easier to improve, and easier to swap out. A drafter that only drafts, using only the inputs it needs, is a better component than a drafter that also does research, checks its own work, and decides when to send.

Narrow by design: the case for composable AI teams covers this principle in depth.

Common mistakes

Skipping the design phase. Teams reach for LangGraph, CrewAI, or AutoGen before they know what the system should do. The frameworks are good. But they don't substitute for design. A badly designed multi-agent system built with a good framework is still a badly designed system.

Over-orchestrating. Not every workflow needs five agents. Some need two. Some need one. Adding agents for symmetry, future-proofing, or because it "feels right" adds coordination overhead without adding capability.

Treating tool access as an afterthought. Tool assignments get made during implementation, not design, and end up messier than they should be. Every agent gets access to everything because it's easier. The blast radius when something goes wrong is large.

No recovery logic. The pipeline works perfectly in the happy path demo and fails silently in production because nobody designed what happens when a tool call returns an error.

Too much context at every step. The full history of the workflow is passed to every agent "just in case." Downstream agents produce worse outputs because they can't distinguish signal from noise. Context should be deliberately scoped, not accumulated.

Designing gates after the fact. A gate added after an incident is a patch. A gate designed in from the start is an accountability structure. The difference shows in how the system handles edge cases and how humans experience the review process.

Design your AI agents in Envelope

Envelope turns a plain-language description of your workflow into a complete AI agent system — agents with named roles, model assignments, tool access, handoffs, and human review gates. Free to start, no code required.

Start designing →

Frequently asked questions

What's the difference between a multi-agent system and a workflow automation?

Workflow automations (Zapier, Make, n8n) connect applications with trigger-action logic. If this happens in app A, do that in app B. They're deterministic, rule-based, and transparent.

Multi-agent systems use AI reasoning at each step. Agents interpret inputs, make decisions, generate outputs, and adapt to context that varies run to run. The intelligence is distributed across the agents, not encoded in the rules of the automation.

In practice, most serious AI workflows combine both: AI agents for the reasoning steps, workflow automation for the deterministic integrations. Envelope compared to n8n, Make, and Zapier covers this directly.

How many agents does a typical workflow need?

Most production multi-agent workflows have three to seven agents. Fewer than three is usually a single-agent task with unnecessary orchestration. More than seven is usually a sign that agent roles aren't clearly defined — you end up with agents that do overlapping things or that exist only to pass context from one place to another.

Start with the minimum number of agents required by the task structure. Add agents when you have a specific reason — a distinct capability requirement, a model routing decision, an accountability boundary — not by default.

What model should each agent use?

Assign models based on the agent's requirements, not the workflow's budget as a whole. Low-stakes, high-volume classification and routing tasks are good candidates for fast, cheap models. High-stakes drafting, analysis, or decision-making tasks are good candidates for frontier models. Specialised tasks (code, legal, medical) may need models with domain-specific strengths.

Model routing in multi-agent workflows covers the full decision framework.

How do I know where to put human review gates?

Put gates before consequential, irreversible actions. Before an email is sent to a customer. Before a CRM record is updated. Before a contract is generated. Before a payment is processed.

A useful heuristic: if a human wouldn't be comfortable finding out the action happened without being told in advance, it needs a gate.

Can non-technical people design multi-agent systems?

Yes — the design work is inherently non-technical. Defining roles, assigning scope, specifying handoffs, placing gates: these are organisational and process design decisions, not engineering decisions. The implementation requires engineering. The design does not.

This is why the design-first approach matters. Non-technical stakeholders can review and approve a team definition before any code is written. Engineers implement from a spec that's already been validated. The feedback loop is shorter and the rework is lower.

What's the difference between an agent and a step in a workflow?

A step executes a predetermined action. An agent reasons about what to do. In a traditional workflow automation, a step calls an API endpoint with fixed parameters. An agent reads its input, decides how to use its tools, and produces output that varies based on what it found.

The line blurs in practice — some "agents" in production systems are closer to deterministic steps with a thin layer of prompt logic. What matters is that the role is clearly defined, the scope is narrow, and the inputs and outputs are explicit.