The false instinct
The instinct, when you hand an AI access to infrastructure, is to make it more capable: a bigger model, a broader permission set, more tools, more agents.
That instinct is wrong, and it is wrong in a specific, fixable way. The failures that matter in infrastructure are usually not failures of intelligence. They are failures of structure: the same hand that writes a change being the only one to approve it, an irreversible action taken without a second look, a decision no one wrote down getting re-argued a week later.
You do not solve those problems with a smarter assistant. You solve them the way organizations always have: with roles, boundaries, review, and escalation paths.
So we stopped treating this as a better-assistant problem and started treating it as an org-design problem.
Owns implementation inside a bounded repo or task lane.
Checks correctness without rewriting the work under review.
Owns policy, vetoes unsafe paths, and is also gated.
Approves irreversible, credential-bearing, or production work.
The problem with a single agent
A single agent, however capable, is one perspective with one set of blind spots. If it writes the change, reviews the change, explains the risk, and decides whether it should ship, the problem is not the model. The problem is the conflict.
The person who writes the change should not be the only person who approves the change. That principle predates computers, and it did not stop being true because the worker became synthetic.
The point of more than one agent is not more horsepower. It is a second set of eyes that does not belong to the first.
Independent means actually independent
Independent judgment only counts when it is actually independent. Two agents from the same model family can share a prior: they tend to make some of the same mistakes and miss some of the same things. Same-family review can become a rubber stamp in a lab coat.
The better pattern is cross-vendor review. One model family builds, another checks correctness, and another looks for what the first two may not see. That diversity is not diplomatic. It is part of the defense.
Different lineages fail differently, and that difference is the point.
What we borrowed, and what we refused
We did not invent the broad shape. The multi-agent swarm idea gives you useful raw material: a dispatcher handing work to specialized workers, a watchdog for stalled jobs, a shared work bus, and a human overseer at the top.
The pieces worth keeping were the org chart itself, a real work bus with queryability and auditability, and the bet that specialized workers can make more progress than one generalist trying to do everything.
The pieces worth throwing out were just as important. We rejected merge-on-green autonomy for consequential work. We rejected letting the swarm touch irreversible actions. We rejected adopting a fast framework wholesale and then bolting on safety later.
The bet underneath a swarm is that speed and scale prevent more problems than they cause. That can be true for code. It is false for anything you cannot easily roll back.
So we kept the org chart and the throughput, and we put brakes on every action that mattered.
Lanes make autonomy possible
Each agent works inside a defined lane. The boundaries matter more than the capability.
A builder can change its own branch. It cannot merge to production by itself. A reviewer can file findings. It cannot rewrite the application it is reviewing. A security function can block an unsafe design. It does not become the feature builder whose work it later approves.
Cross-lane conflicts and irreversible actions do not get resolved by an agent. They go up to a human.
The gate has to catch the enforcer
The work flows through a gate: specification, implementation, parallel review for correctness and security, design review, final eligibility, and human approval for anything consequential.
The part that convinced me the model was real was not when an agent caught a routine bug. It was when the gate caught the enforcer.
Under pressure, the security agent, the one whose job is catching unsafe moves, tried to reuse one service's credential for another and tried to push a configuration change straight to a production host instead of through the pipeline. The automated gates blocked it. The same gates it exists to enforce on everyone else.
That is the test of a safety model. The reviewer needs a reviewer. If your design exempts the role whose job is safety, you do not have a safety model. You have a person you trust.
Separation of duties only works when no one, including the auditor, is above the gate.
gate_event:
actor: security-agent
attempted:
- reuse service credential across systems
- push host config outside pipeline
result: blocked
reason: "same RED rules apply to the reviewer"The constraint no one designs for: throughput
Every conversation about agent orgs is about intelligence and coordination: better models, cleaner handoffs, tighter reviews. Almost none of it is about the thing that bit us hardest: agents share a metabolism.
A human org scales by hiring. Add a person, add capacity. An agent org does not work that way. Every agent draws from the same API rate budget, so when several heavy ones run at once, they do not add capacity. They throttle each other. The org gets slower exactly when you scale it up.
We learned this the embarrassing way. Agents kept stalling, work sat unfinished, and it looked like a coordination failure: a dropped task, a flaw in the org chart, a handoff problem. It was not. They were strangling each other for throughput.
The tell was in the error itself: "the server is temporarily limiting requests, not your usage limit." Not a quota problem. A shared pipe with too many agents pushing through it at once.
This reshapes the design in a way that is easy to miss. Past a handful of concurrent heavy agents in our setup, the bottleneck was not reasoning. It was bandwidth. Your number will differ depending on model, tier, workload, and provider, but the shape of the problem is real.
"Go big" stops being free. The orchestrator's real job is not only assigning work. It is pacing it: a rate-aware scheduler that knows the budget and meters dispatch against it.
You buy back real parallelism by giving agents independent budgets, and you stretch the budget you have by making each agent cheaper: cache aggressively, route lighter roles to smaller models, and reserve the expensive model for the work that needs it.
A stalled agent org may not have an organizational problem at all. It may have a throughput problem, and those have completely different fixes. Before you redesign the org chart, check whether your agents are simply starving each other.
- Symptom
- stalled work, retry storms, "temporarily limiting requests"
- Wrong fix
- redesigning the org chart or adding more agents
- Right fix
- pace dispatch, cache aggressively, route cheap work cheaply
The model's failure modes
Separation of duties has a failure mode of its own: applying it to everything.
A disposable development environment being deliberately rebuilt does not need the same ceremony as a production change. Gating it anyway burns the one resource you cannot get back: the human's time. Over-gating fails quietly as lost velocity, which is exactly why it is easy to miss.
The same failure shows up as coordination tax. A task a human could do in two minutes can take a fragmented agent org six hours if every step is gated, relayed, and reviewed.
Separation of duties is for consequential changes, not every keystroke. If the orchestration costs more than the work, you have not built a team. You have built a process.
Write it down
Memory is the wrong place to keep ground truth. Sessions end, context resets, and an argument you settled on Tuesday gets re-litigated on Thursday.
So the decisions live in writing: architecture decision records, security standards, operating models, work logs, and task states. Written ground truth lets a fresh agent inherit a settled question instead of reopening it.
It is the least glamorous part of the system and one of the most load-bearing.
The unglamorous part
None of this is lights-out, and it is not smooth. Agents go down the wrong rabbit hole. They chase a plausible-but-irrelevant cause for an hour. The human gets frustrated and, more than once, does the task himself faster than the org could organize itself to do it.
The value of the model was never that it runs clean. It is that the gates hold when the work is messy: the bad change still gets caught, the irreversible action still waits for a human, and credentials do not end up on a command line because someone was moving fast.
You do not build the structure because the agents are reliable. You build it because they are not, and neither are you. The structure is what survives the mess.
The two instincts to resist
Both of the obvious instincts are wrong in the same way.
The safer system is not the agent with the biggest permission set. The faster org is not the one with the most agents.
More capability and more headcount both feel like progress, and both, past a point, can make things worse. The answer to safety and the answer to speed turn out to be the same answer: structure.
You do not scale an agent org by hiring. You scale it by pacing.