When Agents Need Help
Part 18 of the Agentic-Oriented Development series. Part III, Chapter 5 of 5.
The Emergency Tree
In December 2004, I was a full-stack developer at a large financial services company in Chicago. The building ran more than forty stories. Trading, operations, and sales all worked out of it, and the executive floor sat on the 33rd. Everything the firm did ran through one address.
That evening I rode the elevator down. By the time I reached the lobby the fire alarm was going off. False alarms happened often enough that I did not think twice. I was already on my way out, so I took off.
Fifteen minutes later the phone rang. The emergency tree had been activated. An electrical fire on the 29th floor had spread to the 30th. The building would be out for weeks. Everything any of us needed was inside it.
Within two hours we were on a group conference call. The Disaster Recovery (DR) plan called for the trading desk to move to a branch overnight. We had one lucky break. A refurbishment project in Michigan was sitting on forty old laptops that had not yet been shipped out. My counterpart in Michigan got in his car that night and drove a box of them to Chicago. They arrived at 11 AM the next day.
My tasks were to install trading software on the branch training-lab PCs, set up the laptops for the operations team that processed paperwork, and pick up our Chief Operating Officer (COO), my boss's boss, in my used Hyundai. I cleaned the car first.
The next morning, trading opened. Of thousands of bank employees, one group was 100% operational. Mine.
Every agent system I sit with now is running an escalation chain like that emergency tree, even when the team has not drawn it. When a subagent fails, when a Model Context Protocol (MCP) server times out, when a guardrail trips, the request travels through a chain of handlers. The shape of that chain decides whether the customer sees one face or sees the failure.
Topology, Protocol Stack, Factory, and Composition each gave you a piece of the architecture. This is the pattern for when a piece fails. The Gang of Four (GoF) called it Chain of Responsibility. The 2004 emergency tree ran the same pattern in phone-tree language. The agent-system version runs across the four layers from Chapter 14 (capability, agent, orchestrator, user), read in the failure direction. The chain held the face the customer saw. That was the whole job.
The Failure Isn't the Worker
The fire was an event. The bank had thousands of competent employees who could keep doing their jobs if they had somewhere to do them from. The failure they had to survive was not that people lost their skills. It was that people lost their workplace, and the playbook ran out before it told them what to do next.
Agent systems fail the same way. The worker behind the orchestrator is usually competent. The model knows the task, the MCP server has the data, and the subagent has the right tools. What brings the system down is rarely that the worker lost its capability. It is that the worker hit a wall and the system did not know what to do next.
The workers are still competent. What changed is the surface they have to work against. Chain of Responsibility is the pattern that names the propagation path. Without a chain, a failed worker is a dead end, and the user sees a stack trace, or nothing at all. With a chain, a failed worker is a handoff and responsibility moves to the next handler instead of stopping at the failure.
The OOP Map: Chain of Responsibility
The Gang of Four described Chain of Responsibility in 1994 as a behavioral pattern for decoupling senders from receivers. A request travels through a sequence of handlers. Each handler either acts on the request or passes it to the next handler in line. The sender does not know which handler will respond. The chain composition can change without changing the sender's contract.
In the object-oriented version, the shape has four parts:
- Handler interface. Every handler implements the same contract. Receive a request. Decide handle-or-pass-along. If pass, hand to the successor.
- ConcreteHandler. A class that knows how to handle some subset of requests. It implements the interface.
- Successor link. Each handler holds a reference to the next handler in the chain.
- Terminator. A handler at the end of the chain that catches anything not handled upstream. Sometimes a default fallback. Sometimes the one that surfaces an error.
The agent version keeps the shape and changes the vocabulary. The handler interface is a contract every node in the chain shares, where each node receives a failure or escalation signal, decides whether to handle it, and if not, passes to the successor. A concrete handler is a specific agent at a specific topology layer. The successor link is the topology relationship that says "if I cannot handle this, here is where it goes next." The terminator is the human in the loop.
Polymorphism from Chapter 5 (the fourth pillar, one interface dispatching to many specialists) made the handler interface possible. Every node in the chain looks the same to the previous node. Each one implements its own decision rule. The orchestrator can route to a different specialist without changing the calling code, because every handler honors the same shape.
The chain runs across Chapter 14's four-layer topology. Those four layers (capability, agent, orchestrator, user) organize every production agent system I review. The capability layer is the lowest of the four. Tools, MCP servers, skills, and the subagents that wrap them live there, doing the actual work.
Each link carries the same three parts, a contract, a decision rule, and a successor to pass to. The 2004 emergency tree ran that structure before the pattern had a name. Each node had an identity, a role, a contact, a timeout, and the next name to call. It was the same handler interface, expressed in phone-tree language.
Holding the Face
I was mid-chain in 2004. I received the call. I executed my nodes. I coordinated with my Michigan counterpart, a peer handler running a parallel branch (the laptop transport). I reported back up the call after the work was finished. The handler that did the work was not the handler with the rank. The chain succeeded because the work-handler had the latitude to act.
Chapter 17 framed how the orchestrator hides the topology from the caller, with the facade and the runtime Strategy that picked the specialist. The chain is what protects that hiding when the topology hits a failure. The face survives because the chain holds it.
The chain holds the face the caller sees. Fixing the failure is downstream of that.
Routing Failure Through the Topology
Chapter 9 framed evals as control flow, the structural reason a probabilistic system needs them to notice when output drifts from spec. An eval is how the system detects that something went wrong. It fires, and then what? The chain is what happens next.
Four default failure paths run through the topology.
Retry the same worker. Cheapest. Idempotent operations (the same operation can run once or ten times with the same result) can absorb retries without compounding cost. Use this when the failure mode is transient, like a rate limit, a model timeout, or an intermittent MCP server hiccup. Each handler in the chain has a retry budget. Once exhausted, the request passes along instead of cycling forever.
Route to a different worker. Strategy from Chapter 17 picks the specialist at runtime by signal. The chain picks the next one when the first fails. The fallback specialist runs the same handler interface with a reduced capability profile. The caller may get a thinner answer. The caller does not get a failure.
Escalate to the orchestrator. When the specialist's local handlers cannot resolve, the orchestrator becomes the decision-maker. It may compose new resources at runtime (the 40-laptops shape). It may consult a peer agent over Agent-to-Agent (A2A) messaging. It may break the task down differently and re-dispatch. Chapter 8 introduced the orchestrator as router. Chapter 17 made it a facade. Chapter 18 makes it a handler in the chain.
Escalate to the user. Terminal handler. The chain ends. The user decides whether to approve a thinner answer, redirect the task, or accept the failure outright.
Three non-negotiables sit underneath those four paths.
Idempotency needs a mechanism, not a claim. A retry that mutates state is not a retry. It is amplification. Saying "the handler should be idempotent" is not enough when the handler is a non-deterministic model invocation. The enforcement surface is the tool layer. Each tool call carries an idempotency key, the orchestrator deduplicates against those keys at its boundary, and the VOICE Principle (Isolated) keeps side effects scoped to a single handler. A handler that cannot produce an idempotency key should not have a retry budget greater than zero. Pass along instead.
Trust boundaries between handlers. Each escalation hop is a forwarded request, which is where the confused deputy (an agent acting on a forwarded request with more authority than it should have) lives. Permissions can be downgraded or preserved across a hop. They cannot be amplified. A handler that gains authority by being escalated to has broken the chain's trust model. The chain becomes a privilege-escalation path.
Escalation budgets are the chain's clock. An escalation budget is a per-handler cap on time, retries, or tokens. When the budget is exhausted, the handler passes along rather than deliberates. A chain where every handler deliberates burns the user's patience and the system's tokens. The 2004 emergency tree worked partly because each node knew its piece and called the next when their piece ran out. Nobody deliberated about whether the problem was really theirs.
Each handler decides fast or passes along. Deliberation is a tax the user pays.
The chain has a telemetry surface that names how an operator sees its health. Per-handler retry count tells you when a handler is burning its budget instead of passing along. Escalation depth from event to terminator tells you when the chain is deeper than the topology should require. Terminator decision latency tells you when the human is being asked to rubber-stamp at machine speed. The three measures catch over-deliberation, infinite chains, and theater terminators before they reach the caller's face.
Graceful Failure and Opportunistic Composition
Graceful degradation, where the caller gets a thinner answer instead of an error, is a different success criterion than "everything worked." The chain succeeds when the caller's face holds.
Chapter 17 introduced graceful degradation as the failure-side variant of Strategy, where the fallback specialist runs the same interface with lower capability, and the user gets a reduced answer instead of a wall. This chapter extends it when even the fallback fails. The chain still has options, but only if its handlers have the latitude to compose new ones.
The 40 laptops were that latitude in action. The trading-desk DR plan had a defined playbook. The operations-team paperwork-processing did not. The chain pulled in 40 work-in-progress laptops nobody had earmarked for the failure mode. The handlers had the authority to do that. Nobody filed a change request. Nobody waited for approval. The Michigan counterpart got in his car. By 11 AM the laptops were on the branch floor.
In agent systems, the orchestrator at an escalation point can compose unexpected resources at runtime too. A peer agent reachable over A2A. A different MCP server with overlapping capability. A skill loaded into a fresh subagent. A cached artifact from a prior session. The chain has the latitude to pull in context that nobody pre-positioned for this specific failure.
Handlers in the chain need permission to compose. A chain that can only route through pre-allocated paths is fragile. A chain that can compose at runtime is resilient. The Governance Triad from Chapter 16 sets the bounds on what composition is allowed at each escalation point, doing its work without the caller ever seeing the contract.
The metric for graceful failure is the caller's face. Did the customer see one response, and did that response carry usable signal? If both held, the chain succeeded. If either gave way, it failed, whether by going silent with no signal or by surfacing a naked error. Either way, the face cracked.
Human-in-the-Loop as Terminal Handler
The chain must end somewhere. A chain without a terminator becomes an infinite-retry loop or a silent failure, and both are anti-patterns. The terminator is the handler that catches anything not handled upstream and forces a decision.
The human-in-the-loop is the canonical terminator in production agent systems. Regulatory environments require it (financial services, healthcare, government). High-stakes environments require it (production deploys, irreversible actions, customer-impacting changes). Even unconstrained environments benefit, because the human terminator is the chain's last sanity check against a silent failure that the automated handlers could not catch.
An effective human terminator has four properties.
The handler receives signal the chain could not resolve. Not a raw stack trace. Not a "please verify" with no context. A summarized escalation that names what happened, what was tried, and what the chain wants the human to decide.
It has the authority to approve, redirect, or accept failure. All three outcomes have to be real options. If the human can only approve, the chain is theater. If the human cannot reject without breaking the system, the chain is theater with a uniform.
It works within a time budget the chain respects. Humans escalate at human speed. A chain that demands a decision in 200 milliseconds has not actually built in a human handler. It has built in a rubber stamp wearing a human's name.
It leaves an audit trail. In regulated environments, the terminator is the signature on the record. What the human saw, what the human decided, when, and on what evidence are all retained. A terminator with no audit trail is a chain with no accountability.
The 2004 terminator was the conference call. Decisions that exceeded any single node's scope went to the room, and the room made the call. The DR plan activation, the branch selection, the resource allocation, the priority order (trading first, operations second) were made at the room level. The chain had a real human terminator and the chain respected the terminator's time. Chapter 9's human grader is a structural cousin, the human in the loop at decision time rather than only at training time.
Chapter 19 takes up "Trust Without Verify" specifically. The rubber-stamp human is its sharpest form, and the next chapter dissects it.
When the Chain Earns Its Keep
A single-agent system rarely needs a chain. Worker fails, surface the error, let the user retry. The chain would be machinery without a job. A two-agent system with visibly different roles may not need a chain either. The failure paths are short. The orchestrator can handle escalation as inline logic without naming the pattern.
The chain earns its keep when four conditions stack up:
Failure modes are heterogeneous. Different failures need different handlers. A retry will not fix a confused deputy, and a fallback will not fix a missing capability. Match the handler to the failure, or the chain routes the problem to something that cannot solve it.
The topology has multiple layers. Capability, agent, orchestrator, user is four layers. A failure at the bottom may propagate through any subset of the others. The chain names the path so it does not have to be improvised under pressure.
Human-in-the-loop is required by policy, regulation, or stakes. Once the human is mandatory at the terminator, the path to the human has to be named. The chain names it.
Specialists have narrow capability and need generalist fallback. Chapter 17's Strategy fallback is the easy case, where a peer specialist covers the gap. When the specialist hits something outside any peer's scope, escalation up to a broader-scoped agent is the only path. The chain provides that path.
Like the topology in Chapter 14, the factory in Chapter 16, and the facade in Chapter 17, the chain earns its place by the constraint, not by the convention. Imposing a chain on a system that does not have those constraints is overhead. Every pattern in this book costs something to maintain, and every pattern repays that cost only when a specific constraint shows up.
Anti-Patterns to Name
The chain has four failure modes that deserve names. Chapter 19 will catalog them inside the broader AOD anti-pattern set. This chapter names them so the reader can recognize the shape when it shows up in a code review or an incident postmortem.
Silent failure. The chain terminates without surfacing anything useful to the caller. The user gets a stub response, an empty answer, or worse, a confidently wrong answer the chain could not catch. The most common cause is a terminator that wraps unresolved failures as success because no path was defined for them. The fix is a terminator that distinguishes "resolved" from "ended."
Infinite chain (retry storm). The chain has no terminator, or the terminator passes along, or the handlers keep retrying without budget exhaustion. Requests cycle, tokens burn, and the user waits. It is the same failure as a TCP retry storm from an earlier era of distributed systems, a feedback loop the system never bounded.
Rubber-stamp human. The human terminator nominally exists but cannot actually inspect, change, or reject the decision. The chain claims oversight. The human supplies a signature. The most dangerous production form is auto-approve-on-timeout, where the chain treats a non-response from the human as approval. The signature is structural fiction. A terminator without authority is not a terminator. It is the appearance of one.
Over-escalation. Every minor failure escalates to a human. The chain becomes a tax on attention rather than a safety net for consequential decisions. The fix is escalation budgets at the lower handlers. Most failures resolve before reaching the human if the lower handlers have the latitude (and the responsibility) to compose, retry, and fall back.
These four are not the full catalog. Chapter 19 expands the picture across the whole AOD methodology and adds the anti-patterns that live outside the chain (monolith agent, vibe coding, and knowledge amnesia, among others). This chapter calls these four out specifically because they live inside the chain mechanism.
What's Next
Part III drew the architecture. Topology gave you the four-layer map, and Protocol Stack gave you the wires between the layers. Factory set the spawn rules, Composition put a facade over the whole thing, and Escalation is the propagation path for when one piece fails. Five patterns, each earning its place by a specific constraint, each one drawable on a whiteboard.
The chain is the last piece because it is what makes the other four survivable. A topology without a chain is a map with no exit. A factory without one ships handlers nobody knows how to fail past. And a facade hides nothing the moment the worker behind it cracks. The chain is what protects the architecture under pressure.
I felt a lot of pride that day in 2004. I will never forget those 24 hours. The pride was earned by adapting the plan, not by following it. That morning, one group out of thousands was operational because the handlers in the chain had the room to act, not because the playbook covered every case. The chain succeeds when the people inside it can compose, decide, and pass along faster than the failure can spread.
Coming up next, Chapter 19 opens Part IV (Practical Wisdom) on the anti-patterns these architecture chapters keep failing on.