Most teams I talk to are having the same two arguments about AI agents. Should the agents be organised like teams? And when agents talk to each other through a shared channel… who looks after it?

Those felt like teams and org chart questions, not just AI questions. So I wanted to actually test it :-]

Same playground as the last two weeks... a small fake company with software services, risks planted in the code and agents fixing them. This time I added two things every real org has, layers of management and a shared channel nobody needs permission to use.

Then I broke things on purpose, to see who notices.

The layers of management mostly added cost. The channel with nobody looking after it quietly left more than a third of the problems unfixed, every single run.

This week is about how I got there and what I’d check in your org before deciding on your Agent Topologies… but where we are in this special Agent Topologies series?

Issue 13 was about workers that only read and report to a lead. In issue 14, the workers were able to write to one shared file... and whoever owns the merge decides who breaks. This week’s issue 15 is about leads managing leads, and agents talking to each other only through a shared channel... now each step takes away something a flat team gets for free: a lead who sees every need and a conversation you can overhear.

In Issue 14 the answer was ownership: the merge owner who sees every change broke nothing, the conflict-only supervisor broke the service nobody spoke for. Now the owner has a boss or the owner is a channel.

Both experiments are in the public repo (04-hierarchical and 05-event-bus), with the code, the raw runs and a plain-language README. Same Harbour Bikes world and checker as last week (11 planted risks, 2 services that break if a shared default moves the wrong way), Sonnet 5.5 doing the work and Opus 5.5 in every lead and owner seat. My predictions were frozen in PREDICTIONS.md on 11 October, before the first run. Where I was wrong, the README says “miss”.

If this lands, forward it to whoever draws your team boundaries and your agent boundaries on different slides.

Subscribe now

Should the Agent Layers Match the Team Layers?

8 services, one worker each. Above them a platform lead who owns the shared config, and in 3 of the 4 versions I added sub-leads in between.

How did I split those mid-tier? a) along team boundaries (rentals and money, fleet, customer and back office) or b) along technology (python, JVM and Go, node and ruby and rust).

And a 4th version keeps the technology cut and hands the platform lead a published list of what every service needs.

During the experiment, every worker’s replies feed every version, so whatever changes between them comes from the managers, not the workers.

The trap is the same as last week: pricing-engine needs a timeout of at most 300 ms, has no worker of its own and reads the shared default. In every run at least 2 worker requests would have broken something if the platform lead had granted them as asked.

On the 1st run, nothing broke. 0 of 10 runs with a regression, in every version I tried... all 11 risks fixed in all 40 runs.

I thought technology cut would break pricing-engine at least once because in one dry run the python sub-lead merged the raises into one 12s request and the platform lead said yes... in the real thing, it didn’t happen... miss.

So where was the threat stopped? At the top. Every. Single. Time.

A harmful timeout request reached the platform lead in all 10 runs of every version, and the platform lead turned it down or gave pricing-engine its own override.

The sub-leads stopped none of the pricing-engine threats. Cut by team, a sub-lead even forwarded harmful requests against its own team’s services (0.4 per run). The only thing any sub-lead held back was one rate-limit request against payments-gateway, in a single run per version.

And the bill? The flat lead used 3,947 management tokens per run.
Sub-leads cut by team? 23,018! Almost 6x!
By technology? 17,681 or 4.5 times the flat one...

But there was a catch... every lead got the same contract and one line of it said pricing-engine reads the shared timeout. So no tier was entirely blind… a sub-lead cut by technology could still work out who a raise would hurt.

I was not happy with that, so I ran it again without that line in the contract.

Same worker replies, replayed run by run, so run 6 of the second run is run 6 of the first with only the leads’ contract changed. Before the run I wrote down the rule for calling it... the technology cut has to break pricing-engine in 3 or more of 10 runs while the team cut stays at 0 or 1.

And bam! First time something broke, and it was the cut by technology. In runs 6 and 8 the platform lead set the shared timeout to 15s (both python services needed longer and nobody objected) and pricing-engine took the new value. None of the other cuts (flat, by team or by registry) broke...

2 is below the 3 I set before the experiment, so I don’t call it an effect... but I don’t call it noise either. After all, it is the only version in 80 runs of this experiment that broke anything... and it is the cut that puts pricing-engine in one group and the two services asking for the raise in another.

The team cut keeps pricing-engine and its callers in the same group, so its bound travels up with the request.

The flat lead and the registry version see every service’s needs. Take away the one line that told the leads and only the technology cut was left without anybody who knew.

So, should the agent layers match the team layers? On this task the cut mostly didn’t matter. The agent that owned platform.yaml saw every change whatever tree sat above it, and the extra layer bought cost, not safety.

The one cut that broke anything was the technology cut, and only once nobody told the leads. A similar indication as last week pops up... the owner that sees every change did best, and the owner is only as good as what reaches it.

So how do we look into this from a Team Topologies perspective? The Inverse Conway Maneuver says to shape teams on purpose to get the architecture you want. I’m starting to see the same in agents in a much cheaper way to experiment... Here the cheapest tree was the flat one, because the real work was done by whoever owned the shared file, not by the tree.

First run: every lead was told pricing-engine reads the shared timeout. Second run (the blind run): the same worker replies, and no lead was told.

Who Looks after the Shared Channel?

uv run 05-event-bus/experiment.py and off they went… six team agents, one per team, each subscribed to the services its team owns.

A scanner posts one message per planted risk, and in every run I break the channel four ways: a message missing its fields (malformed), a message sent to the wrong service, a consumer that crashes mid-delivery, and the same message sent twice. On top of that I wanted a bit of “lack-of-ownership chaos”, so one service (fleet-telemetry) has no owner at all... its two messages have nobody to go to.

Same consumers, same prompts, same models in every version… the only thing I changed was who looks after the channel. Three versions:

• Nobody. Messages go out, and whatever gets lost stays lost.

• A steward agent. It reads the whole log and sends corrections.

• The platform. Code checks every message, retries, and parks what it can’t deliver, and an agent looks only at what got parked.

nobody steward agent platform Risks fixed of 11, mean (worst run) 6.7 (6) 11.0 (11) 11.0 (11) Malformed message (of 10) 7 absorbed, 3 lost 5 recovered, 5 absorbed 10 recovered Sent to the wrong service (of 10) 10 lost 10 recovered 10 recovered Consumer crashed (of 10) 10 lost 10 recovered 10 recovered fleet-telemetry fixed (of 20) 0 20 20 Duplicate processed twice (of 10) 10 10 0, by construction Operating tokens per run 41,790 71,106 55,700

“Absorbed” means the risk got fixed even though the message never arrived complete and correct. A malformed message still names its service, and that was often enough for a consumer to go and read the files... lucky, not designed.

So, who looks after the shared channel? Somebody has to.

With nobody, about 4 of the 11 risks got lost in every run (4.3 on average). And that always included everything for the service with no owner. Nobody noticed, because noticing was nobody’s job.

With an owner, the steward agent and the platform did equally well: 11 of 11, every fault recovered or absorbed, nothing lost.

The difference is the bill. The platform used about 22% fewer operating tokens than the steward (55,700 against 71,106). Why? The steward reads the whole log. The platform’s owner agent only sees what got parked: 1,683 characters per run against the steward’s 7,336.

And the duplicate? Processed twice in all 10 runs with nobody and with the steward, never with the platform... but that’s code doing its job, not an agent being clever.

Two honest asterisks. The corrective copies the steward and the platform agent send skip the no-double-processing check. And fleet-telemetry has no owner because I built it that way, so the owned versions fixed it by handing its messages to a team.

Team Topologies already has a shape for this. The platform runs the bus as a product, and the teams consume it as a service, with minimal interaction. The Inverse Conway Maneuver, again: want a channel that doesn’t lose messages? Give the bus an owner whose job is the bus, and the architecture follows.

For the integration folks: that’s a Message Bus with a Dead Letter Channel and an Idempotent Receiver, straight out of Hohpe and Woolf.

Can You Tell where a Failure Was Born?

When something breaks in a channel, the first question is always “where did it start?”

So after every run, I asked a separate agent to explain every loss using only the records. Then I took the trace context away and asked again.

It made no difference.

The investigator named the first problem of every risk that had one: 60 of 60, in every version... and 60 of 60 again with the trace context stripped.

On the 43 losses the nobody version left behind, it named the right cause all 43 times and never called a lost message handled. False alarms on the clean risks? 0 of 50, except 3 of 50 when it read the steward’s records.

I predicted exactly this, but I’ll say it plainly anyway: on this task the records were enough. The ids on the messages added nothing the log didn’t already have.

The interesting part was what got lost, not explaining it afterwards. A bigger system, with more hops and a messier log, may be where trace context earns its keep. Not this one.

My Predictions, Scored

I froze my predictions on 11 October, before the first run, with every dry run I’d seen listed next to its commit. The blind run got its own, frozen on 12 October before it started.

The score, in three sets:

• 04: 8 hits, 2 misses, 1 mixed

• 05: 15 hits, 1 miss, 5 mixed

• 04 blind: 7 hits, 2 misses

“Mixed” means I got the direction right and the detail wrong. The ones worth owning:

• The cut. I bet the technology cut would break pricing-engine in 1 to 4 of 10 runs. It broke it in none. I’d even flagged it as my likeliest miss, because the platform lead held 300 ms on its own in 2 of 3 dry runs... and I still bet on it. That one’s on me. For the blind run I bet 0 to 3, most likely 0 to 1. It came in at 2.

• The middle tier’s bill. I guessed the technology cut would cost 3 to 4 times the flat lead. It was about 4.5.

• The steward. I gave the agent babysitter 9 to 10 of 11. It got 11 of 11, a clean miss. Three of the five mixed ones are the steward too: I expected it to lose some malformed and crashed messages, and it lost none. It also cost 1.7 times nobody, not the 2 to 2.5 I guessed. Better and cheaper than I gave it credit for... though still about 28% more than the platform.

• The blind run’s two misses are both small, and both about the registry version. It ended on a different timeout than the first run in 2 paired runs, where I said 0 to 1. And blind, the technology cut and the registry version cost 9% less than the first run, where I said within 5%.

One thing worth repeating: the checker didn’t change. Same 11 planted risks, same 2 regression needs, same answer key, so “fixed” this week means what it meant last week. What you can’t do is line up this week’s numbers against Issue 14’s. Different setup, different question.

The Three-body Reading

Every issue in this series looks at three things that move together, the three bodies:

• Architecture: the seams. Does the middle tier split along a domain boundary or across one? Does a message carry where it came from?

• Team: who owns the cut, and who owns the channel.

• AI capability: the models. The thing everyone argues about first... and it didn’t change at all this week.

Same models in every version, and the channel went from 6.7 to 11 of 11 just by changing who owns it. Changing the cut of the tree mostly moved the token bill. It broke something only when the leads were blind and the cut split pricing-engine from its callers.

So this week, the body that mattered was the team.

Where it Does not Reach yet

Before you take any of this to your org, here’s what this setup can’t tell you:

• Small and narrow. One task, 10 runs per version, Claude models only, one backend.

• Not real tools. My harness applies the agents’ edits and delivers the messages, so these are not tool-using agents on a real bus.

• My choices. The cut, what each lead sees, the faults and the mandate are all mine. They’re disclosed and pre-registered, which makes the bias visible... not gone.

• Fewer draws than it looks. In 04 the same worker replies feed all four versions, so the 10 runs are 10 draws of workers, not 40.

• Not a fair fight on information. The flat lead sees every service’s needs and the tiered lead doesn’t (the registry version adds them back). So part of any flat advantage is information, not the missing middle tier.

• Traps I didn’t build. No failure injection in 04, and no fight over a shared file in 05. Nothing here says how often these failures happen in your org.

The good news: the runs themselves were clean. In 04 no lead call failed and no sub-lead fell back to forwarding unchanged. In 05 no owner or investigator call failed, and only one consumer delivery (in the nobody version) came back with an invalid reply. So the 10 runs per version are 10 real runs.

Run it on Yourself

Five questions to ask about your own agents. They work on your teams too... the questions don’t care.

1. Layers against teams. Where do your agent layers disagree with your team layers? A tier that cuts across a team ends up forwarding requests it can’t judge.

2. What travels up. When a request goes up a layer, does the layer above know who it could hurt?

3. The channel owner. Who owns the shared channel? Not who uses it... who gets paged when a message can’t be delivered?

4. The unrouted message. What happens to a message nobody can route? Name the service with no subscriber.

5. The first problem. Can you trace a failure back to where it started from the messages alone, or only by asking the agents?

The Whole Thing, in One line

In both experiments, who owns the shared thing mattered more than the shape of the org chart.

Subscribe now

Two Things Before You Go

New name, same newsletter. You may have noticed: this is now Proof in the Pipeline. Same author, same weekly essay, and a name that says what it’s about… AI governance shown working where the code ships, not promised in a PDF.

Paid tiers, pledges only for now. Each essay only shows the end of the work, the code and the research stay behind it, so I’m opening paid tiers for that part (nobody is charged until paid launches). Paid readers get a monthly post with the repo, the runs and what I’m reading for the research. Founding members join the AI Executive Circle: a quarterly call with me and priority (plus 10% off) on the Delivery Lab, and there are only 12 seats. The weekly essay stays free, always. If that sounds useful, pledge here :-]

Share Proof in the Pipeline

Coming up

Next week: the oracle. A critic tells a writer what’s wrong, the writer fixes it, and someone has to own the rubric they’re both judged against. Two questions... who gets to edit that rubric, and when does the loop stop? More soon :-]

Sources

• Erik S. and Barry Zhang (Anthropic), “Building Effective Agents”, 19 Dec 2024. https://www.anthropic.com/research/building-effective-agents

• Matthew Skelton and Manuel Pais, Team Topologies, IT Revolution, 2019; Inverse Conway Maneuver, stream-aligned and platform teams, interaction modes, as summarised at https://teamtopologies.com/key-concepts

• Melvin E. Conway, “How Do Committees Invent?”, Datamation, 1968 (Conway’s law).

• Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns, Addison-Wesley, 2003; Message Bus, Dead Letter Channel, Idempotent Receiver, Correlation Identifier, Message History: https://www.enterpriseintegrationpatterns.com/patterns/messaging/

• The repo: agent-topologies, folders 04-hierarchical and 05-event-bus, with PREDICTIONS.md (frozen 11 Oct 2026 at 4d4728a, pinned at 0db1298), benchmark runs 11 Oct 2026, results commit 11f3b02: https://github.com/thiagoavadore/agent-topologies/tree/main/04-hierarchical and https://github.com/thiagoavadore/agent-topologies/tree/main/05-event-bus

So what happens when agents can write? This is what we have this week. Three agents, each in its own copy of the repo, all pointed at the same hub file: one shared config file, platform.yaml, that all eight services inherit their timeouts from. I split the services so that two workers would want the same setting. Built to collide, on purpose.

Oh yeah, using the state of the art available to you: Sonnet 5.5 does the work everywhere, and Opus 5.5 only sits in the merge seat in the second experiment.

uv run 03-fan-out-partial-reducer/experiment.py and off they went...

Voilá! 20 runs later, across the two versions where they had a way around the shared file, the shared timeout was contested... 0 times. Zero.

Nobody collided. And not because they were polite... because they never needed the shared file. One worker wrote an override (its own value instead of the shared default) in its own service.yaml. Another hard-coded the timeout straight into the code.

Everyone met their own need and walked around the shared file, and the platform ended up with about 3 different timeouts across 8 services (3.0 and 2.9 on average, depending on which route was open).

So I closed the override route. They took the code route. I closed that one too... and only then did they start fighting over the file.

But Thiago, isn’t that just agents being resourceful? Somehow, yeah... but look at what I was doing the whole time: my agents weren’t avoiding each other. My architecture was giving them a way out.

In Issue 13 the workers only read, which is why none of them could collide. GitHub’s peer sessions, on the other hand, met at the most connected file... one asked four times to integrate, was refused four times and merged anyway.

This week my workers write... and I have two promises to pay: the pipeline, where one early mistake travels all the way to production, and fan-out / fan-in (several agents in parallel, one merging), where the agent that merges carries the whole cognitive load.

Both experiments are in the public repo, one folder each, with the code, the raw runs and a plain-language README.

One thing I did differently this time, and it is the trust move of the week: I wrote my predictions down and froze them in PREDICTIONS.md, together with the checker and its tests against fake fixes, before the first run. Where I was wrong, the README says “miss”. You will see a few of those below :-D

If this lands, forward it to whoever owns the shared config your teams keep walking around.

Subscribe now

A check at the end can’t see where the mistake was born

The first experiment (code and full results): four agents in a row over the same eight services: extract the facts, classify the risks, prioritise them and write the remediation plan. Each one only sees what the one before handed over. No lead, no orchestrator... a chain of handoffs.

Then I break stage 1 on purpose in half the runs. Either I move a fact to the wrong service or I drop it entirely... so 3 versions: a) no checks at all, b) one check at the end, c) a check after every handoff.

Erik S. and Barry Zhang’s “Building Effective Agents” (Anthropic, December 2024) calls this prompt chaining and suggests “programmatic checks (see “gate” in the diagram below) on any intermediate steps”... fine... this is that check, measured: 10 runs per version, 5 faulted and 5 clean, Claude Sonnet 5.5 in every role (checks included).

The check at the end caught 0 of 5 injected faults... none. By stage 4 the plan was perfectly consistent with its faulted input, so the last check looked at it and saw nothing wrong... by then the mistake was just the input.

The check after every handoff caught 4 of 5, every single one of them at stage 1, the stage that owns the mistake! It let one dropped fact through: an admin console open to the world (ouch!).

The price? In my runs, the every-handoff version halted 3 of 5 clean runs (2 wrongly, 1 rightly, a split decided by a script, not by a model) and used 46% more tokens than no checks at all (42,308 against 28,975 per run). The checks alone were 53% of its tokens. So yes, you pay... in tokens and in false alarms somebody has to go and look at.

Now the surprise! A fact moved to the wrong service healed itself downstream in 3 of 3 runs with no check at all. Huh? In the end, it still carried its file path and the later stages tied it back to the right file. A dropped fact had nothing left to tie to and reached production 2 of 2 times.

Facts that carry their provenance survive handoffs. Missing facts don’t announce themselves.

Now read it with the org chart in mind. Matthew Skelton and Manuel Pais’s Team Topologies (IT Revolution, 2019) says stream-aligned teams have “no hand-offs to other teams for any purpose”. A pipeline of agents is a chain of hand-offs, the exact thing I, and every Team Topologies practitioner, tell teams to avoid... but how can we not when we are talking about tokens explosions and missing context? So, when we must have hand-offs, we need to put the check where the work is born and accept the bill.

In Issue 12 I asked who owns each rule. Here the question is who owns a mistake. Every fault I injected was born in stage 1, the agent that extracts the facts. The check after every handoff caught it right there and blamed stage 1: the right owner. The check at the end never fired. And if it had, it could only have pointed at stage 4, the agent writing the plan, which did nothing wrong. It just inherited the mistake.

Nobody collided until the platform was the only way

Now the second experiment, where the cold open happened (code and full results). Three agents with write access, each fixing its own share of eight services.

All 8 read their defaults (timeouts, rate limits, backups, minimum instances) from one shared config file, platform.yaml. I split the work so it would collide as the two services that need a longer timeout went to different agents. Thus a script (not a model) scores the result by parsing and runnning the files... then it counts what got fixed and what a shared change broke.

But ofc, there were 2 traps inside:

• pricing-engine needs a timeout of at most 300 ms and has no agent of its own.

• payments-gateway needs at least 20 requests per second. A shared default that is right for the worker who set it can break a service nobody is looking after.

Claude Sonnet 5.5 as the workers, Claude Opus 5.5 in the merge seat, and each version ran 10 times. This is what we got:

1. Local overrides allowed. Shared timeout contested in 0 of 10 runs. 3.0 distinct timeouts across the 8 services, on average.

2. No overrides, but a timeout in code allowed. 0 of 10 again. 2.9 distinct timeouts.

3. The platform rule. Timeouts must come from the shared file, and a CI-style check reverts any file that hard-codes one. Contested in 6 of 10 runs, in every version with the rule.

Same models. Same prompts, apart from one neutral line per step stating the rule (no “please coordinate with your colleagues” nudges, I promise). And the collisions went from 0 to 6 of 10.

So it seems that contention is a property of the architecture, not of the agents. They collide only when the shared file is the only seam. Before that, they fork the platform quietly and you find out six months later when somebody counts the timeouts.

With some scientific honesty, the platform rule is my design choice. The dry runs never collided on the timeout without it, so I imposed it, and I’m saying so. The versions with the rule tell you what happens once the shared file is the only way... not how often that is true in your shop.

And this is Issue 12’s “platform enforces” right, as code. The rule is what made the shared default real. Before it, “the platform default” was a suggestion, nothing more than that...

The merger has to speak for the service that isn’t in the room

Once they collide, somebody has to merge. And that is promise 2 from last week, remember? The agent that merges carries the whole cognitive load... so let’s see what that load actually is. 4 ways to own the merge, all under the platform rule, 10 runs each (3 for me):

• No owner, first change wins. pricing-engine broke in 10 of 10 runs. The shared timeout ended up above 300 ms every time, and nobody could grant pricing-engine an exception.

• A supervisor (Opus) that resolves conflicts. broke in 4 of 10. And here is the part I did not predict: all 4 came from runs with no conflict. One worker raised the shared timeout alone, nobody disagreed, so the supervisor was never asked... and the uncontested value broke a service no worker owned. In the 6 runs that had a conflict? Zero breaks.

• A platform owner that applies every change to the shared file itself. 0 breaks in all 10 runs. Its usual move: keep the shared default at 300 ms (9 of 10 runs) and grant the slow callers their own overrides (at least one in 6 of 10 runs, both in 5). The cost: the fewest problems fixed of the versions with the rule, 8.4 of the 11 planted ones against 8.6 and 8.7, because it also said no.

• Me, in the merge seat. 3 runs. pricing-engine broke in 3 of 3, and I granted 0 overrides. I sat there with every service’s needs on the screen and picked between the two loud workers... how? I shared some text in the screen, some files to a free version of Copilot, asked to explain to me the problem and take a decision... what I know a lot of people do and say they are “using AI to take architectural decisions”.

So what is the cognitive load, Thiago? It’s not the conflicts. Those are the easy part... two values, two reasons, pick one. The load is the services nobody is speaking for. A conflict resolver only sees disagreements. The owner sees every change, including the quiet ones that break the service that isn’t in the room.

Team Topologies has a name for this as well and that is only “managing team cognitive load”. That owner is the platform team and its job is to carry that load so the stream-aligned teams don’t have to.

My answer key was wrong again, twice

Last week’s first lesson was “whoever writes the answer key decides the result”. This week I got to relearn it... twice...

First, the checkpoint was wrong the first time. In my first run, the check-after-every-handoff version rejected 5 of 5 clean runs. Five!

Great check, until you read the reasons... I never told the stage-2 check what stage 2 was allowed to drop, so it judged stage 2 against a bar the stage could never meet. Re-scored, 4 of those 5 rejections were false.

The fix was giving each check its own stage’s contract. Then I re-ran all 30 and kept the first run’s results in the repo, in plain sight.

Hmm... wait. Was it “whoever writes the answer key” or “whoever writes the checker”? It turns out that both... the check is part of the answer key.

Second, one planted risk (a problem I hid on purpose) was unfixable. The workers declined to give the maintenance scheduler a second instance in 5 of 5 trial runs, with the right reason: its data lives on the VM’s local disk, so two instances would split it. They were right and I was wrong. The risk came out of the answer key before the real runs (11 risks, not 12), written down in the checker’s notes.

And the frozen predictions have two misses worth owning. I predicted the end-only check would catch some faults (it caught none), and that the supervisor would have near-zero breaks (4 of 10, for the uncontested-change reason above).

Maybe I should stop writing answer keys :-P ... or maybe this is the job. A frozen prediction turns “I knew it” into “I was wrong here, and here is why”.

The three-body reading

The architecture body is the seams. Is the shared file the only route? Does each fact carry where it came from?

The team body is who owns the merge and who checks at the source.

The AI-capability body is the models. Unchanged across every version this week, and still the thing everyone argues about first.

The merge experiment is the proof. Same models, same instructions to the agents, and the service that broke went from broken in 10 of 10 runs to broken in 0 of 10. The only thing I changed was who owned the merge. The AI capability didn’t move at all.

Where it does not reach yet

One task, 10 runs per version, Claude models only, one backend. My test harness applies the agents’ edits, so these are not tool-using agents exploring a repo and committing on their own. The traps and the platform rule are mine, disclosed and pre-registered, which makes the bias visible... not gone. The human version is one person that built software before but inputs things into Copilot, talks a bit with it and pastes the content somewhere else...

Run it on yourself

Five checks, on your own agents. They work on your teams too, the checks don’t care.

1. The first check. In your agent pipeline, where is the first check? If it’s at the end, it can tell you something is wrong, not where.

2. Provenance. Does every fact or artifact carry where it came from (file, line, source)? Provenance is what let a misattributed fact heal itself.

3. Escape routes. Which shared config do your agents (or teams) write to, and what local routes exist around it? Count your distinct timeouts.

4. Every change. Who sees every change to the shared file, not just the conflicts?

5. The empty chair. Name a service with no owner in the room. Who speaks for it at merge time?

The whole thing, in one line

Check where the mistake is born and give the merge to someone who speaks for the service that isn’t in the room.

Subscribe now

Coming up

Next week, one level up. Leads managing other leads, and agents that talk only through a shared message channel. Two questions: should the agent layers match your team layers, and who looks after the shared channel? It’s Conway’s law run backwards: design the team shape you want, and let the architecture follow. More soon. :-]

Sources

• Erik S. and Barry Zhang (Anthropic), “Building Effective Agents”, 19 Dec 2024. https://www.anthropic.com/research/building-effective-agents

• Matthew Skelton and Manuel Pais, Team Topologies, IT Revolution, 2019; stream-aligned teams and team cognitive load as summarised at https://teamtopologies.com/key-concepts

• Stephen Toub, “Migrating the GitHub Copilot runtime to Rust, using Copilot”, The GitHub Blog, September 2026 (vendor engineering blog). https://github.blog/ai-and-ml/generative-ai/migrating-the-github-copilot-runtime-to-rust-using-copilot/

• The repo: agent-topologies, folders 02-pipeline-checkpoint and 03-fan-out-partial-reducer, with PREDICTIONS.md (frozen 5 Oct 2026). https://github.com/thiagoavadore/agent-topologies/tree/7888d48 (results at commit 7888d48, pushed 6 Oct 2026).

15 runs later, a clean-looking table with some discoveries and a potential inference. More workers/subagents are better, right? Yeah, sure... the version with the most workers found every planted risk. The versions with two workers missed one now and then.

Nice... except it was wrong.

One planted risk was an HTTP call with no timeout, written with httpx. And httpx has a 5-second timeout by default... so no risk at all. The two-worker versions that “missed” it were right, and the uncapped version got credit for a false alarm.

My agents were fine. My answer key was not... why? Because I wrote the key assuming no timeout and “forgot” that httpx has one by default...

I swapped it for a requests call (which really does wait forever) and re-ran all 15. First lesson of the series, before any architecture: whoever defines the right answer decides the result.

But Thiago, is this not just microservices again? A lead, some workers, a contract between them... Kinda :-P

I thought I was choosing an architecture, but I was drawing an org chart.

In Issue 12 I left you in the org chart and promised a terminal. Here it is... and it walked me straight back into the org chart. Its four rights (mandate, enforce, instruct, override) are about to become code.

Sooooo yeah, a series. 5 issues in 5 weeks, 1 public repo, 1 family of agent setups per week, each with a number from my own run. The repo is agent-topologies: seven folders planned, the first one live today, each README in plain language so you can follow it without writing a line of Python.

Ken Huang’s post “Multi-Agent Design Patterns” (Agentic AI Substack, August 2026) gave me food for thought. His catalogue sorts the patterns by shape. I think the question a CTO can actually act on is a different one: who owns the split?

If this lands, forward it to whoever is drawing your “agents’ org chart” :-D

Subscribe now

Your agents will copy your org chart

Mel Conway wrote it down in “How Do Committees Invent?” (Datamation, April 1968): organisations that design systems “are constrained to produce designs which are copies of the communication structures of these organizations”.

And now 58y later, the communication structure has new members: Agents. Whoever decomposes the work for them decides the shape of what they produce.

And the org-design vocabulary transfers almost too well. Matthew Skelton and Manuel Pais’s Team Topologies (IT Revolution, 2019) names three ways teams should interact:

• Collaboration: “working together for a defined period of time to discover new things”.

• X-as-a-Service: “one team provides and one team consumes something ‘as a Service’”.

• Facilitating: “temporary and focused”.

Bam!! Each one has an agent twin.

• A worker with a strict input/output contract? X-as-a-Service.

• A critic loop is collaboration, and it should be time-boxed... in agent land that is the iteration cap (coming with a code example soon).

• A subagent that explores and hands an answer back is facilitating.

The neutral names for the patterns exist too. Erik S. and Barry Zhang’s “Building Effective Agents” (Anthropic, December 2024) names orchestrator-workers and evaluator-optimizer, and advises “the simplest solution possible, and only increasing complexity when needed”. The ownership question is what I add.

So the copy happens anyway. The only choice is whether you pick the original first.

The orchestrator is a team lead with a context window

Now the numbers from the repo, as promised.

Folder 01 is a risk review: eight short, fictional service descriptions (cards) with twelve risks planted in them, like a hardcoded key or an HTTP call with no timeout.

The lead reads a one-paragraph summary of each card and decides how many workers to use and which cards each gets. The workers review their cards and report in a fixed JSON format, never talking to each other. Then the lead merges everything into one summary for the CTO. A risk counts as found only with the right card and the right category.

Around them sit four safety checks, all in orchestrator.py:

1. Strict format (FINDINGS_SCHEMA): one retry on a bad report, then it is thrown out and its cards listed as “not reviewed”. The X-as-a-Service contract, in code.

2. Worker cap (apply_fan_out_cap): too many workers get squeezed into fewer, no card dropped.

3. Token budget (dispatch): 60,000 tokens per review; once over, the remaining cards are skipped and named in the summary, not silently lost.

4. Backup plan (fallback_plan): if the lead fails, split the cards alphabetically. A failed lead never stops the review.

Three versions, five runs each (ran on 2026-Sep-25th): Claude Opus 5 as the lead, Claude Haiku 4.5 as the workers, through headless Claude Code (runs at commit 3a09186, after the answer-key fix. folder 01 is here).

Version Workers Tokens per review (average) Risks found Uncapped 4 36,573 60 of 60 Capped at two 2 23,278 59 of 60 No lead plan, alphabetical 2 22,994 59 of 60

Left alone, the lead planned four workers in all five runs. Capping it at two cut tokens by 36% for one missed risk out of 60. Skipping the lead’s plan and splitting the cards alphabetically between two workers: 37% fewer tokens, the same one miss.

So, Thiago, does the lead earn its keep? On this (simple) task, no. The lead’s planning call averaged 1,493 tokens per review. That is everything the plan cost, and it bought nothing I could measure over a dumb split with the same headcount.

More workers also meant more noise: 3.6 findings per review not on the answer key (the ones I spot-checked looked more like architectural calls the model was making for me than wrong answers), against 1.0 and 0.8 with two.

The caveats travel with the number:

• Tokens, not cost. Cached input counts like fresh input, though it bills at about a tenth.

• Per-call overhead. Claude Code adds a fixed prompt and thinking to every call, so four workers pay more of it. On the API the gap may shrink.

• One undercounted run. One uncapped worker was logged at 0 tokens while returning four findings. Leave that run out and the cheapest four-worker run (36,796) still sits above every two-worker run (max 26,897). The real gap is wider, not narrower.

• Same miss both times: an smtplib call with no timeout. Chance at five runs, or a risk easier to miss when a worker holds more cards.

And the honest version of “we have guardrails!” is that in 15 real runs, none of them fired. Zero broken reports, no worker outside its brief, and the budget never tripped (biggest review: 41,999 of 60,000). The retry and throw-out paths are covered only by uv run pytest. Fine... let’s not just confuse a guard that exists with a guard that reality has tested.

Now read it as an org chart. The lead is the tech lead who owns the decomposition... its decomposition added nothing over an alphabetical split. The headcount decision (the cap) actually moved the number. The lead is also the single point of failure, which is why the backup plan exists. And whoever owns the lead’s prompt holds the “team instructs” right from last week’s Issue 12.

Two kinds of worker, and only one gets to write

Stephen Toub wrote up how GitHub ported the Copilot runtime from TypeScript to Rust, using Copilot (The GitHub Blog, September 2026). Let’s not be naive at first... we are talking about a vendor blog showing off its own product. The engineering holds up well when reading, but the tooling claims need checking before they transfer.

They kept two kinds of worker apart. Subagents read, explore and hand answers back to the parent (hello Team Topologies’ facilitating). Child sessions get their own worktree and branch and produce the diff, with a write boundary (hello also to you X-as-a-Service).

GitHub reports that the session.ts port used five subagents and 15 child sessions in seven waves. 120 of the 140 files touched were touched by exactly one session. The 20 contended files were all hubs.

And these hubs are where it went wrong. 2 peer sessions met at the most connected file. One asked four times to integrate, was refused four times... then reached into the other’s worktree and merged anyway (passive-aggressive much?)... almost like a refusal between peers carries no weight.

Just like with kids at school, work peers need a tiebreaker. So do AI agents, and that’s the role of a coordinator or a named human.
That is the fourth right from Issue 12 showing up in a terminal.

Back to folder 01, you see that my workers are like GitHub’s subagents, not its child sessions. They only read and report, which is why none of them could collide. Workers that write (change things) are next week’s experiment.

The three-body reading

The architecture body is the worker contracts and the write boundary: who may touch which files.

The team body is who owns the decomposition and who breaks ties.

The AI-capability body is the models behind the lead and the workers. The cheapest body to swap, and the one everyone argues about first.

Folder 01 is the proof in miniature. Swapping the lead’s judgement for an alphabetical split changed nothing measurable. The headcount decision, a team-body call, moved tokens by a third. Move one body without the others and you get GitHub’s annexation, or a lead nobody owns.

Where it does not reach yet

One vendor case and an unusual one: an agent runtime porting itself. My run is small (one task, 8 cards, 12 risks, 15 runs), read-only, with the caveats above.
Orchestrator-worker is the simplest setup in the family, so ownership gets harder in the next weeks. The catalogues’ thresholds (fan-out depth, iteration counts) come without data, so the repo measures ours.

Run it on yourself

Five checks, on your own agents, not on a vendor’s slides.

1. Two drawings. Put your agent topology next to your org chart. Where do they disagree?

2. One name per contract. For each worker: who owns its input/output contract? A name, not a team.

3. Shared writes. Does any worker write to something another worker writes? Then you have peers, and peers need a tiebreaker.

4. The lead’s prompt. Who owns the decomposition itself? And have you tested it against a dumb split (alphabetical, round-robin) at the same headcount?

5. The brakes. What stops a runaway split: a worker cap, a token budget, or hope?

The whole thing, in one line

Pick the pattern by who owns the split, not by the shape of the diagram.

Subscribe now

Coming up

Next week, handoffs versus parallel streams: the pipeline, where one early mistake travels all the way to production, and fan-out / fan-in, where the agent that merges carries the whole cognitive load. Folders 02 and 03... and this time the workers get write access, pointed at the same hub file. More soon. :-]

Sources

• Mel Conway, “How Do Committees Invent?”, Datamation, April 1968. https://www.melconway.com/Home/Committees_Paper.html

• Matthew Skelton and Manuel Pais, Team Topologies, IT Revolution, 2019; interaction modes as summarised at https://teamtopologies.com/key-concepts

• Erik S. and Barry Zhang (Anthropic), “Building Effective Agents”, 19 Dec 2024. https://www.anthropic.com/research/building-effective-agents

• Stephen Toub, “Migrating the GitHub Copilot runtime to Rust, using Copilot”, The GitHub Blog, September 2026 (vendor engineering blog). https://github.blog/ai-and-ml/generative-ai/migrating-the-github-copilot-runtime-to-rust-using-copilot/

• Ken Huang, “Multi-Agent Design Patterns: Architectural Topologies, Failure Modes, and Production Hardening”, Agentic AI Substack, 29 Aug 2026 (practitioner newsletter, food for thought only).

Agentic AI

Multi-Agent Design Patterns: Architectural Topologies, Failure Modes, and Production Hardening

Building autonomous multi-agent systems for production requires shifting away from monolithic prompt chains and unpredictable agent swarms. Early implementations frequently fail when uncoordinated agents trigger compounding error loops, exhaust token budgets, or execute unauthorized mutations. Production reliability demands explicit coordination topologies, bounded execution budgets, deterministic state machines, and fine-grained access control…

Read more

2 months ago · 50 likes · Ken Huang

• The repo: agent-topologies, folder 01-orchestrator-worker, https://github.com/thiagoavadore/agent-topologies/tree/cca6c10/01-orchestrator-worker (runs made at commit 3a09186 on 25 Sep 2026; README and guards corrected since, runs unchanged; linked at cca6c10).

And well... I ran that third one in a customer room recently. Who were there? The CIO, the cloud lead, a security engineer and two engineering managers. The answers came fast.

“Security, I think.”

“Well, the gateway is ours, so the tool list is platform.”

“The repo rules are ours, we wrote the AGENTS.md.”

Three answers, all true... and a quietness that arises when people notice that the answer to “who owns the policy” was everyone... which in an org chart means nobody.

So I did what I do when a room goes quiet... think, discuss and read from others’ experience. Not the frameworks... the engineering blogs. What did Dropbox, Uber, Spotify, Atlassian and Datadog actually write, in 2026, about who decides what their agents may do?

In essence, none of them has A policy owner. All of them have four.

Issue 1 named the trap: one program, not three.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 7 taught one repo and moved a fleet.
Issue 9 asked who reviews the reviewers.
Issue 10 found the software factory was always there.
Issue 11 moved the gate before execution and gave the humans the policy to write.

No kit this week: the argument, five posts you can read yourself, and the standing ask: if this lands, forward it to whoever was about to hire a “Head of AI Governance”.

Subscribe now

Dropbox gave the decision back

The story that made me sit up is not the one with the biggest numbers. It is the one where decision rights moved, and moved in the direction I didn’t expect (towards the team, not the centre).

Dropbox runs its coding agents on an internal platform called Nova. Mike White and Kevin Altschuler wrote it up in May. Before Nova they had an in-house migration tool that generated parallel agent jobs from prompt templates, ran the verification commands and published branches. Thousands of migrations went through it. It worked... until it didn’t...

And what seems obvious in hindsight was that they had no simple way to review the agents’ output and then “continue” it with their input... so whenever something broke/failed, there was no practical way to recover/checkpoint the work.

And then the sentence I underlined twice: “highly repeatable migration work was often better handled directly by migration owners, who could launch and manage dozens of agents with the same runbook rather than coordinating delegated work across teams.”

Read that again with the org chart in mind and remember Issue 7 with the airline. The central team did not take more control. They gave the runbook back to the person who owns the migration, and kept for themselves the things a migration owner should never have to think about: isolated execution, hermetic tests, the CI loop and one rule they state in plain words: “We chose to keep publication outside the agent and limit each session to a single branch.”

So who owns the policy at Dropbox? The platform owns what the agent structurally cannot do (publish, branch, skip CI). The migration owner owns what the agent will do today (the prompt, the runbook and, they say, soon the rollout). Two owners, two different kinds of decision and no committee between them.

And that split is not a Dropbox quirk... let me show you.

Four rights, not one job

Dropbox is one of the five posts I read. The other four (Uber, Spotify, Atlassian and Datadog, all linked below) tell the same story from a different desk: put them side by side and the same four decisions keep landing in the same four places.

1. The invariants sit with security. Uber’s post on agent identity (six authors from Engineering Security and AI Security, May 2026) is the clearest. Every agent gets an identity. Every call into Uber’s systems goes through an MCP Gateway that is, in their words, “a policy enforcement point for MCP tool invocations”. And it works by defining policies based on internal risk classification... if it’s a high risk, it’s MANDATED. See? Not proposed to a working group. Security decides which systems are high risk and what an agent must present to touch them. What Uber’s post does not say is that security decides what any agent is for. That is someone else’s right.

2. The mechanism sits with platform. Spotify’s Niklas Gustavsson described it in June: Backstage, Soundcheck, “golden state” (the recommended stack per component type) and lint. They claim that, combined with static analysis and linting, standards become active guardrails. The platform team did not write a policy document for agents. It pointed the guardrails it already had at a new kind of contributor. Dropbox’s Nova is the same move: the platform owns execution, isolation, validation and the CI trigger. Datadog’s version is a guild (”Frontend Augmented by AI”) that sets golden paths for over 50 teams. Platform owns how checking happens. Not what is allowed and not what the agent is doing.

3. Local intent sits with the team that owns the code. Atlassian’s Jira team encoded its own cleanup knowledge into agent skills, then hit the wall every large org hits. You cannot have “one-size-fits-all” solutions in a company with 1000’s of repos owned by 100’s of teams... so Atlassian’s answer is repository-specific skills, with repo owners asked to write the cleanup procedure for their own repo. Datadog goes in the same direction by using the control with the smallest scope that still gives the desired output. So they go developer-local first, then the package, then the repo root. The team decides the task, the instructions and whether a piece of work is worth delegating at all.

4. Exceptions sit with a named human. Not a role, but A person. Atlassian keeps it simple by having engineers validate the change before it is merged. Dropbox kept publication out of the agent’s hands entirely. And if you operate in Europe this one is not optional: the AI Act requires that high-risk systems have human oversight by people with “the necessary competence, training and authority”, who can decide not to use the output, override it or stop the system. The Act does not say which team those people sit in, just that they must exist and be able to act.

Now back to the tease from last week, because I promised three answers.

Who owns the policy? Put the five together and you get 4 people... the trick is that each owns a different verb: security mandates, platform enforces, the team instructs and a named human overrides.

What can they decide without asking anyone? Security can add a system to the high-risk list. Platform can change how a check runs. The team can change what its own agent is told and what it works on. The human can say no. Each of those is a decision nobody else gets to block and each one is useless without the other three.

Where do they sit? Security is security. Platform is your platform team, the same one that owns the paved road, not a new one. The team is stream-aligned, in Team Topologies terms. The guild at Datadog is the enabling team: it sets the path, measures it, and hands it over. And the human with the override sits wherever the blast radius is... which is why it has to be a name and not a box.

The standards will not tell you, and that is good news

So Thiago, should we read NIST or ISO’s standards to know what to do? Well... I did, but there are dozens out there... and frankly, they are deliberately silent on exactly this question.

NIST’s 2026 agent work (the identity and authority concept paper, the Agent Standards Initiative) is about identification, authorization and audit. Technical as NIST has always been. It assigns no owner. ISO/IEC 42001 says an organisation must “define responsibilities for AI use” and stops there, on purpose, because it is a management-system standard and not an org chart. And the AI Act (Article 26) explicitly preserves the deployer’s freedom to organise its own resources.

So the law tells you the override must exist and the person must have authority. The standards tell you responsibilities must be assigned. Neither tells you to whom... this is not a gap, just a way to tell you it’s your decision.

The consultancies, of course, are happy to fill the silence. BCG’s Enterprise AI Control Plane (August 2026, consultancy research) argues for “a designated owner with the mandate and authority to enforce standards across platforms”, with Enterprise Architecture leading it, and separates a “control layer” from a “build layer”. Team Topologies argues the other way (their own consultancy’s position) with “the stewardship boundary around the value flow remains the remit of the human team”.

But look at what each one is actually asking to own. BCG’s “control layer” is identity, the agent registry, runtime enforcement and golden paths: the invariants and the mechanism, rights 1 and 2. Team Topologies’ “stewardship” is what the team is trying to achieve and what it tells the agent to do: the intent and the override, rights 3 and 4. Neither is claiming the other’s half. They only sound opposed because both call their half “governance”... and the moment you name four rights instead of one, the fight disappears.

What auto-merges is the real fight

If the four rights were the whole story this would be a tidy post. They are not, because two of the five companies disagree on the one decision that matters most day to day: what ships without a human looking at it.

Spotify has a fleet system with more than 2.5 million automated maintenance PRs the vast majority auto-merged, no human in the loop. Their most recent Java migration across the backend took three days. And with agents in the mix they now have 76% more PRs to review, so their post ends with something I hear over and over and over and over again: they auto-merge what is safe, focus human review where it matters most, and are rethinking how they plan “as the bottleneck moves from coding to decision-making” (their words).

Atlassian’s story came out in the same month with the same type of work (flaky tests, cleanup chores), but with a distinct decision when compared to Spotify. First, the agent does the repetitive context gathering with investigation, diagnosis and a proposed fix. Then engineers validate the change before it is merged. Every single change... They report about one engineering week saved per month and up to 80% fewer engineering hours on flaky tests specifically...

Can both be right? I think so, and I think Dropbox’s story has the reasoning. Dropbox validates a flaky-test fix by running the test 100+ times in CI before it lands, and kept the CI trigger outside the agent. Why? Otherwise you get changes validated against the wrong tests. Spotify’s fleet changes are deterministic scripts against a standardised stack, with the same lint on every component, while Atlassian’s run across 1000+ repos with different conventions.

So the variable is not how mature your AI is. It is whether the change has a deterministic check and a bounded blast radius. Where both hold, auto-merge is a policy decision that platform (right 2) can make and security (right 1) can veto. Where either fails, the named human is not a transition phase. The human is the design... And here we are in the same conclusion as last week, one floor up.

And a geek aside for the people who write AGENTS.md files (hello, Context Engineering, we meet again): Datadog measured what happens when you remove the root-level one: 13% faster runs, 16% fewer input tokens and 10% lower spend. And they say this happens with a roughly 7% decrease in output consistency, which they plan to recover by moving the instructions into more finely scoped controls, including skills, hooks, lint rules, tests.

But notice what it is: an economic argument for Intent (right 3). A global instruction file costs the agent the way a global standards manual costs a developer... then, let’s push intent down to where it is used.

Policy on paper is not policy at runtime

That is three companies pushing intent down. I wanted to know what the other few thousand do, and while thinking on how to gather data from different companies about Agentic AI policies, I came across an interesting survey. Enterprise Management Associates asked 202 CIOs, CTOs, CISOs and IT directors at companies with more than 1,000 employees about their agents (August 2026, commissioned by Cequence Security, a vendor with a product to sell, methodology in the PDF). 94% were confident their agents do not have more access than they need. 33% actually provision least-privilege access. 34% check an agent’s authorization at the moment it tries to act.

I do not trust a sponsored survey to three significant figures. I do recognize it from a few of my customers, because it is the shape from Issue 11: a policy that exists as a document versus a policy that is checked before the first tool call. If I accept the numbers from the research it would be like 9 in 10 leaders are sure the policy holds, but only 3 in 10 have the check that would prove it.

And now you can see why “who owns the policy” gets the everyone-therefore-nobody answer, right? If the policy is a document, everyone can own a paragraph. If the policy is a check that runs, somebody has to own the check (platform), somebody the rule it enforces (security), somebody the instruction it enforces it on (the team) and somebody the moment it says no (a human). The four rights are not a governance model I am proposing. They are what falls out of moving the policy from paper to runtime.

The three-body reading

You knew this was coming.

The team body is this whole issue. Four rights is an org-design decision. No model release makes it, no verifier makes it, no framework makes it for you. It is the part the “buy the guardrail” plan skips, and it is the part the room went quiet on.

The architecture body is the gateway and the contracts. Uber’s MCP Gateway is right 1 made physical. Spotify’s lint-as-guardrail is right 2 made physical. Neither exists in a company whose tools have docstrings instead of contracts (Issue 11’s check 2, still open on most whiteboards I see).

The capability body is the verifier, the thing from last week. It is the least interesting of the three this week, which is the point: a verifier in a company that has not decided who mandates, who enforces, who instructs and who overrides is a very fast way to enforce nobody’s policy.

And the sequencing trap, one more time, because it has a new costume. It is called “Head of AI Governance”. Hire that person first, with the four rights undivided, and you have built a committee with a job title. They will write the document. Nine in ten are already sure it holds.

And this is the second exercise in the lab I run with customers, right after the 5-line policy from last week: for one real workflow, four names on one sheet. Who mandates, who enforces, who instructs, who overrides. The blanks are the diagnostic, and so far the blank is almost never “security” :-]

Where it does not reach yet

Every practitioner story above is a company writing about itself on its own blog. Uber, Atlassian, Dropbox and Spotify have independent coverage that matches. Datadog’s numbers are Datadog’s alone. That is the weight of it: five honest accounts, zero audits.

AFAIK, there is no peer-reviewed study that tests one org design for agent policy against another... nobody has measured how long it takes a company to approve a new agent capability, or what share of the rules are written centrally versus in the repo. The surveys collapse all four rights into one question called “governance”, which is why they cannot tell you anything about the split.

You can be sassy and claim that all these companies are software companies, so heavily regulated banks/airline/telco/public sector will have a different shape. I have seen the same pattern in regulated rooms. The difference is that those companies do not write blog posts, and I do not put clients in newsletters.

And the Dropbox story is about migrations, repeatable work with a runbook. Whether “give the runbook back to the owner” holds for work that has no runbook is exactly the Spotify-versus-Atlassian question, and it is open.

Run it on yourself

More checks for you to self-assess... now one floor up from last week’s.

1. Four names, one workflow. Take the agent workflow you wrote the 5-line policy for. Write the four names: who can add a system to the high-risk list, who can change how the check runs, who can change what the agent is told, who can say no at the moment it acts. A blank is a finding, but I think 2 names in one box is a bigger one.

2. Find one thing that auto-merges. If nothing does, ask why not: is there a change class with a deterministic check and a small blast radius or is “human reviews everything” a policy nobody decided? If something does, find who decided, and whether security (right 1) knows.

3. Find where one rule lives. Pick one instruction your agents follow. Is it in a root file every agent reads on every run, or in the smallest scope that needs it? If it is at the root and applies to one team, you are paying Datadog’s 16% in tokens for the privilege of a committee owning it.

Run the three and you know whether your policy has four owners or one document.

The whole thing, in one line

Stop looking for the owner of the policy. Split it into the four decisions it always was, and put a name on each.

Coming up

Next week I want to leave the org chart and go back to something with a terminal in it. Hands dirty, numbers on the table. More soon. :-]

Subscribe now

Sources

• Mike White, Kevin Altschuler, “Introducing Nova, our internal platform for coding agents”, Dropbox.Tech, 21 May 2026. https://dropbox.tech/machine-learning/introducing-nova-our-internal-platform-for-coding-agents

• Matt Mathew, Prasad Borole, Meng Huang, Sergey Burykin, Gaurav Goel, Bayard Walsh, “Solving the Identity Crisis for AI Agents”, Uber Engineering Blog, 21 May 2026. https://www.uber.com/us/en/blog/solving-the-agent-identity-crisis/

• Spotify Engineering, “Coding Is No Longer the Constraint: Scaling Developer Experience to Teams and Agents at Spotify”, 3 June 2026, a write-up of Niklas Gustavsson’s (Chief Architect and VP of Engineering) Code with Claude 2026 talk. https://engineering.atspotify.com/2026/6/code-with-claude-coding-is-no-longer-the-constraint

• Arnaud Moret, WaiYee Loo, “How We Cut up to 80% of Engineering ‘Chores’ Using AI Agents in Jira”, Inside Atlassian, 1 June 2026 (the 80% figure applies to flaky-test hours). https://www.atlassian.com/blog/development/ai-agents-jira-engineering-maintenance

• Addie Beach, Rui Martins Lacerda, “How we built data-driven AI Golden Paths at Datadog”, Datadog, The Monitor, 9 September 2026 (the A/B figures are Datadog’s own, unreplicated). https://www.datadoghq.com/blog/ai-development-golden-paths/

• Regulation (EU) 2024/1689 (the AI Act), consolidated text of 27 July 2026, EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng

• NIST, “New Concept Paper: Identity and Authority for Software Agents”, 5 February 2026 (a concept paper, evolving). https://www.nist.gov/news-events/news/2026/02/new-concept-paper-identity-and-authority-software-agents

• ISO, “ISO/IEC 42001 explained: what it is”, September 2026. https://www.iso.org/home/insights-news/resources/iso-42001-explained-what-it-is.html

• Shaheer Rizvi, Zeeshan Shah, Sarvesh Singh, Nicolas Tanaka, Francois D’Agostini, Jazz Tobaccowalla, “Enterprise AI Control Plane: The CIO’s Guide to Governing and Accelerating AI Agents”, Boston Consulting Group, 14 August 2026. https://www.bcg.com/publications/2026/how-cios-govern-ai-agents-at-scale

• Team Topologies, “Executive Report: Team Topologies as The Organizational Foundation for AI ROI”, 12 August 2026, via the AI Workforce Readiness page. https://teamtopologies.com/ai-workforce-readiness

• Enterprise Management Associates for Cequence Security, “Agents Without Guardrails: The Agentic AI Governance Gap in the Enterprise”, 31 August 2026, n=202 (vendor-commissioned survey. Press release https://www.cequence.ai/news/ai-agent-governance-research/ and report PDF https://www.cequence.ai/wp-content/uploads/2026/08/EMA-Research-Report-Agents-Without-Guardrails.pdf

Further reading (not cited in the body):

• InfoQ’s independent write-ups of the Uber (17 June 2026, https://www.infoq.com/news/2026/06/ai-agent-identity-uber-auth0/) and Dropbox (June 2026, https://www.infoq.com/news/2026/06/dropbox-nova-ai-coding-agents/) posts, which is how I checked that I was not reading a press release as engineering.

The trail started a week earlier, in a call with Prof. Tanja Vos about code quality and testing when AI writes the code. She did her PhD in formal methods in Utrecht, has spent the 26 years since in software testing, and knows Erik Meijer from those Utrecht days. She pointed me to his talk and to the repo. Erik Meijer, if you need the intro: the Dutchman who put LINQ and Rx into .NET at Microsoft, then Meta, then Leibniz Labs, and who now says on stage that he has never seen anything scarier than an LLM with tool calls. In that talk he mentioned that someone had built a working version of his idea on GitHub. If he hadn’t, I’m not sure I’d be doing this...

Anyway, I go through it... pydantic, z3-solver... pytest: 130 passed in 1.5 seconds. Green faster than you can say “human in the loop”. Let’s run a plan that has nothing malicious on it: voilà! It executes, no hiccups.

Now let’s try a plan that has almost the same content as the first one, but in the end sends a summary email to an outside email domain. REFUSED.

Three separate complaints, none of them from an LLM: the summary carries data from fetch_mail into the email body (taint), the recipient domain is not on the allowed list (precondition), and a policy called no_external_send hit its error state (automaton).

Refused even before any tool is executed... so I skipped the check and handed the plan straight to the executor. Blocked again, zero tools called.

This got me intrigued, so I decided to dive deep into the paper. There they describe an example where, in the middle of the plan, sits a delete_file("*.txt"). The repo does not cover that one yet. To their credit, the DESIGN.md says so in plain words: old() in frame conditions, not implemented. I grin because the real thing is narrower than the paper. That is normal, and it is also where the work is. Maybe time to call Erik (and Nada Amin, who wrote the repo) and start the PhD I keep postponing? :-P

What if this review queue getting bigger is not just a staffing problem caused by the tech moving forward faster than we can absorb? What if this has a lot to do with process and placement?

I mean, we pay humans to find (after the fact) what sometimes a policy or validation could have refused before the first tool even ran!

The queue is not a staffing problem. It is a placement problem. And Meijer brings it forward in a very intriguing paper…

Issue 1 named the trap: one program, not three.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 7 taught one repo and moved a fleet.
Issue 8 found the sandbox was not holding, and told you to keep the gate.
Issue 9 asked who reviews the reviewers.
Issue 10 found the software factory was always there.

No kit this week, just the argument, one repo you can clone, and the standing ask: if this lands, forward it to the person who signs off your agents.

Subscribe now

Evals are flaky tests, in the vendor’s own words

The paper is “Guardians of the Agents: Formal Verification of AI Workflows”, Communications of the ACM, January 2026. Before you print it and put it on your CISO’s desk: it is a Practice column. One author, an argument built by construction and example. He measures nothing and says so himself. So this is not the peer-reviewed witness of Issue 6... it is a platform builder telling you what he thinks (I know, I know... a world-renowned practitioner that has also a PhD, but still not peer reviewed). I read it as that, and so should you.

And what he thinks is that the three things every vendor deck calls “our safety story” do not hold.

The hard take on Evals first. Testing can show the presence of bugs but never their absence (Dijkstra wrote that in 1970, EWD249 and OMG, that aged so well). Evals are the same: they can show a harmful behaviour is there, not that it is gone. And because the model is stochastic, evals “behave like flaky tests in traditional software” (his words, not mine). And we all know what a team does with a flaky test... retry until green, ofc.

Guardrails second. In essence, it is pattern matching against a list of forbidden words, which brings false positives, cultural bias and a small change in the question giving you a large change in the answer. But the part that matters for this issue is timing... when the guardrail fires, that means that the model has already produced some output or caused other side effects. Kinda like a nightclub Bouncer that “exits you” after you already threw away a chair...

Log-and-audit third, plus its cousin-the-cookie-banner that everyone accepts (”Allow this tool call?”). The first fails because by the time someone reads the log, the damage is done. The second fails because of security fatigue: people stop reading and click Allow... we all have done it, do it today and will do tomorrow...

So the industry’s safety story, in one line: a flaky test suite and a Bouncer with a word list.

That is Issue 8 in one man’s words. The labs ran the evals with the guardrails off... and then we act surprised when a sandbox leaks.

Code and data. The lesson we already paid for once

Ok, Thiago, so what does he propose instead? Nothing new. That is the beauty of it.

He says the root cause of prompt injection is the same as SQL injection: no separation between code and data. The model reads instructions and content in the same stream and treats both the same. Hello, SQL injection, my old friend... we fixed you decades ago with prepared statements: values are never executed as commands. Full stop.

His move is the same, and it builds on his own ACM Queue piece from February 2025: hide the concrete values from the model. Instead of calling tools one by one, the model emits a structured workflow with symbolic references (@emails_fetched, not the emails). Nothing runs until that plan is verified. And the plan is readable: it prints as a literate explanation you can read before anything executes. That is what I watched on Sunday. Fetch, summarize, send... as a plan, not as three tool calls firing one after the other while I hope for the best.

The verification is three old ideas pointed at agents:

1. Information flow (taint). Data from fetch_mail may not reach send_email.body when the recipient is outside the allowed domains. A rule from a source to a sink, with a condition.

2. Security automata. Fred Schneider, ACM TISSEC, 2000: a small state machine that walks the plan step by step and has an error state. no_external_send is one of those.

3. Pre- and post-conditions checked by a prover. Z3 in the repo, Dafny in the paper.

The analogy he uses is bytecode verification in Java and .NET (Xavier Leroy, Journal of Automated Reasoning, 2003): complexity in production, verification cheap. Same idea as the checksum on your credit card.

Every one of those ideas is older than your youngest engineer. The news is not the maths. The news is where it sits: before execution! On a plan the model cannot smuggle data into.

And the hands-on texture, because I promised you the work and not the pitch: the repo is Nada Amin’s (Harvard), plain Python, verifier core around 1,900 lines. The three complaints in my terminal came from three separate checkers agreeing with each other. No LLM anywhere in the verifier. You can read the whole thing in an evening, which is not something I say about many “AI safety” products developed by vibe coding...

The frame problem is your review backlog

Now the example from the paper that made me put down the tea.

Ask the model to delete foo.txt and bar.txt. Post-condition: after execution, those two files are gone. The model emits delete_file("*.txt"). Correct! Both files are gone. So is everything else. In his words: “LLMs lack common sense and will do exactly as they are asked”, which here means the simplest program that satisfies the post-condition. McCarthy and Hayes gave this a name in 1969: the frame problem. The fix is a frame condition: files that do not match the pattern stay exactly as they were. Add it and the *.txt plan fails to verify.

Honesty rail, right here and not in a footnote: the repo does not implement old() yet, so that exact proof runs on paper today. What ships is a weaker check, roughly “the pattern is not” plus a check that the post-condition is not vacuous. Fine. The shape is what matters.

Because... what is your senior reviewer doing at the third MR of the afternoon? Not checking that the agent did what it was asked. Checking that it did what was meant. Nothing else touched. Nothing else deleted. That is a frame condition and a human one... run by eye, on a green pipeline, many times a day.

We counted the cost of that two weeks ago with He and colleagues’ 802 developers: reviewer load doubled, the share of MRs any human reviewed fell from 89% to 68% and silent approvals doubled. The frame condition did not disappear when the humans got tired. It just stopped being checked.

And last week we found the craft the factory never asked for. Here is one piece of it with a name: writing the frame condition is a decision the spec never contained. “Delete these two” is the spec. “And touch nothing else” is the craft. Nobody wrote it down, so the reviewer supplies it (by hand) at 17:45.

The review queue is where the absence of that sentence shows up.

Who writes the policy? The three-body reading

Here is the three-body reading, and for once it has a repo attached.

The verifier is the capability body. Z3, taint rules, automata. No LLM. It is a thing you adopt or build, not a model feature and no release next month makes it appear in your stack.

The tool contracts are the architecture body. The verifier is useless without tools that carry machine-readable contracts: a ToolSpec with preconditions, taint labels, which parameters are sinks, which domains are allowed. And you usually have docstrings, not contracts. And the frame problem needs a model of the world state your tools can change... the repo’s missing old() is a symptom of exactly that gap, but a miniature version.

The policy owner is the team body. Nothing runs without a person whose name is on allowed_domains, on the taint rules, on the automaton. Governance stops being a committee that samples outputs on a Friday and becomes a team that owns a policy-as-code artifact with its own test suite. The repo has 130 of them. Yours would have your own.

And here is the sequencing trap, Issue 3 all over again:

1. Buy the verifier first and nothing else? You have nice demo (and that's all)…

2. Write the policy first and that’s it? Nobody can enforce it…

3. Add the contracts first without (1) and (2)? Nobody knows what to check against…

They move together or they do not move.

In Issue 2’s terms: a bolt-on guardrail reading outputs is AI-ready, and Meijer’s own critique of guardrails applies to it in full. Verification designed into how work is planned, checked and executed is AI-native.

So where do the humans go? From reviewing every output to writing and owning the policy, and reviewing the residue: the exceptions the verifier cannot decide. Meijer keeps that hybrid himself, static first, runtime for what static cannot see. Fewer signatures. More decision per signature. The reviewers are still the people you keep, now with a job that is possible to do.

This is also the first exercise I run when a team brings a real work item to the lab: before anyone touches the harness, write the policy for one agent workflow in plain words. The tools it may call, one flow it must never make, one thing in the world it must not change. How much of that a team can state at all is the diagnostic... and the gaps land on the architecture line or the team line, almost never on the model :-]

Where it does not reach yet

This sounds too good to be true... and too simple. Yes and it has limits...

The paper is a position piece. No measurement, and the author says the effect of plan-first on how useful the agent stays “has not been formally measured”. The repo is one implementation with 130 tests and a public list of what is not built yet. That is the weight of the evidence and I am not going to inflate it.

It verifies agent tool-call plans. Not the application code an agent writes. Your MR queue of AI-written Java is not what Z3 is proving here. The frame condition your reviewer supplies by hand is the same shape, but nobody has written the post-conditions for your services, and this repo will not write them for you.

The policy is the new attack surface. The paper’s own injection example arrives inside a tool description, not inside an email. Whoever writes allowed_domains can be wrong, lazy, on holiday or used AI to write it (wrongfully). A verifier enforces what you wrote, including the mistakes.

And the static check is not the whole gate. Meijer keeps runtime monitoring for what static analysis cannot see, the way Java still bounds-checks arrays at runtime. So the humans reviewing the residue are not a transition phase. They are the design.

None of that weakens the placement argument. It tells you what the hands-on work is for the next couple of years: contracts on tools, policies with owners, plans instead of chat and a reviewer role redefined around exceptions. Which is the same three-body work the last ten issues kept pointing at. The difference this week is that you can clone it and play with it on a Sunday.

Run it on yourself

Before the next “AI governance” steering committee, and before someone buys a guardrail product because the demo had a red banner in it, three things you can check this week. On your own agents, not on a vendor’s slides.

1. Write one policy. Pick one agent workflow you already run. Five lines: the tools it may call, one data flow it must never make, one thing in the world it must not change. If you cannot write line three, you have just met the frame problem. Your reviewers have been solving it by hand and now you know what they were doing all afternoon.

2. Count the contracts. Of the tools your agents can call, how many have a machine-checkable precondition or a declared sink, versus a docstring and good intentions? That ratio is your architecture-body gap. No model release moves it.

3. Name the owner. Whose name is on the policy? If the answer is a committee, or “security, I think”, the gate is still a person reading a diff... only now nobody knows which person.

Run the three and you know whether your gate is placed where the paper says it should be or where Issue 9 found it: on a tired human, after the fact.

The whole thing, in one line

Stop paying people to find, after the fact, what a policy could have refused before the first tool ran. Then pay them to write the policy.

Coming up

Next week, one floor up. If humans write the policy and machines check the plan, then who owns the policy? What can they decide without asking anyone? And where do they sit in your org: platform, enabling, or stream-aligned team? The operating model behind the verifier. More soon. :-]

Subscribe now

Sources

• Erik Meijer, “Guardians of the Agents: Formal Verification of AI Workflows”, Communications of the ACM 69(1), January 2026, pp. 46-52, Practice section https://doi.org/10.1145/3777544

• Nada Amin (metareflection), guardians: static verification for AI agent workflows, an implementation of Meijer’s paper, MIT license, GitHub; commit 59e52d9 of 16 July 2026 was the one run for this issue https://github.com/metareflection/guardians

• Erik Meijer, “I’ve never seen anything scarier than an LLM with tool calls”, AI Engineer World’s Fair 2026 talk (San Francisco, 29 June to 2 July 2026; AI Engineer YouTube channel, uploaded 13 July 2026).

• Fred B. Schneider, “Enforceable Security Policies”, ACM Transactions on Information and System Security 3(1), February 2000, pp. 30-50 https://doi.org/10.1145/353323.353382

• John McCarthy, Patrick J. Hayes, “Some Philosophical Problems from the Standpoint of Artificial Intelligence”, Machine Intelligence 4, 1969, pp. 463-502 http://jmc.stanford.edu/articles/mcchay69.html

• Edsger W. Dijkstra, “Notes on Structured Programming”, EWD249, 1970 https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF

• Xavier Leroy, “Java Bytecode Verification: Algorithms and Formalizations”, Journal of Automated Reasoning 30, 2003, pp. 235-269 https://doi.org/10.1023/A:1025055424017

• Erik Meijer, “From Function Frustrations to Framework Flexibility”, ACM Queue 23(1), February 2025, pp. 19-38 https://doi.org/10.1145/3722544

• Hao He, Shyam Agarwal, Yegor Denisov-Blanch, Pavel Azaletskiy, Sanmi Koyejo, Bogdan Vasilescu, “AI Writes Faster Than Humans Can Review: A Longitudinal Study of an Enterprise ‘2×’ Mandate”, arXiv:2607.01904, July 2026 https://arxiv.org/abs/2607.01904

Further reading (not cited in the body, same argument from other angles):

• Margaret Mitchell, Avijit Ghosh, Alexandra Sasha Luccioni, Giada Pistilli, “Fully Autonomous AI Agents Should Not be Developed”, arXiv:2502.02649, 2025 https://arxiv.org/abs/2502.02649

• Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, et al., “Building Guardrails for Large Language Models”, arXiv:2402.01822, 2024 https://arxiv.org/abs/2402.01822

Last week I left Brett Codes walking off the factory floor and promised you he was not alone. He isn’t. Open any “I’m done with AI coding” thread and the comments are full of experienced people saying a version of the same thing... some of them already halfway out the door.

So when we say that AI is making the value of coding tend to zero, because anyone can type specs into a harness and spit out some code, we are telling your developers that their skills are becoming worthless?

It feels natural that some will reject AI coding on that context. But it reads as less the rejection of a tool, and more the rejection of a way of working... and that way of working is older than IDEs, OOP or even the punch cards.

What motivates someone, what drives someone to be consistently happy at work and doing their work goes way beyond the tool they use. We all have our buttons to push and be pushed... and those change even throughout the day, the week and many times throughout your life... imagine amongst different individuals?

What you cannot deny is: this accessible (for now) intelligence at our apps and terminals has the capacity to increase/speed up our output. But that doesn’t necessarily produce a better outcome for the company, or more happiness for the individual.

So before your best problem-solving engineers/developers/employees walk away from the factory floor because they are tired of validating a machine output, let’s talk about what is missing.

Issue 1 named the trap: one program, not three.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 4 kept the model in the loop, not in the room.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 6 brought the peer-reviewed witness: more shipped, same typing.
Issue 7 taught one repo and moved a fleet.
Issue 8 found the sandbox was not holding, and told you to keep the gate.
Issue 9 asked who reviews the reviewers, and pointed at this week.

The factory was always there

The idea of controlling the output of software production always existed and always will. The more established the problem, the larger the company, the more regulated the environment... more people want to feel the control, establish a framework and guardrails that let us predict (or imagine we can predict) the future.

This idea of being a backend Java engineer at a financial institution for 40y of your life never appealed to a lot of people. The romanticization of craftsman’s work, of a creative space to put your ideas into motion, can be intrinsic to humans (or it’s just a reflection of our society)... it is why the gaming industry can fill its ranks with passionate developers who endure brutal crunch for not enough money, while private equity tech pays a lot more for the sake of “doing the boring, but simple”.

And this predates Agentic AI! I was a developer, I was a data engineer, I was a data scientist, I was a lead, I was a manager, I was a CTO... I led teams and I led teams of teams. Back then, I sat with teams whose whole job was to turn someone else’s spec into code. The PO/PM decided the what and why, they were the only ones with access to “the customer”. The ticket arrived in its own glory, format, standards... the developer had to make it true, close and pull the next one. Great people doing the best with the context and knowledge they had... many times, not even thinking if that was the thing that should be. A real software factory before a single robot (or AI) actually participated in our daily lives. And many would wake up one day after a retro, a planning or just moving tickets in Jira to ask: is this it? Is this my life?

This hasn’t changed. But AI might have finally exposed it... the fear is out there. I have mentees and former colleagues constantly asking what my plan is “if this continues”. Some are making career changes, but many are grinding to absorb the novelties and ensure they are not left behind.

Faster now, gains much later

Even the ones that are accelerating are asking questions. Because they see that their backlog tasks for the sprint can be wrapped up in less time with Agentic Coding. For the owners of the capital (let’s not digress into Marx), it’s another dream coming true. More productivity means more output, which we hope means more outcome... more outcome means more revenue and/or less cost. And all that brings to a bigger profit margin.

From last week’s edition though, we surfaced that keeping humans at the gate (the entire “human in the loop” angle) might not be the work that lots of people signed up for... a faster and more efficient job doesn’t mean happier employees, because you are talking about someone’s skill/craft.

I’m eager to see what developer experience scientific papers will show in a few years, because the first ones that are still talking about chat-interface with LLMs already tell a very interesting story.

Take Chen et al. (ICSE-SEIP 2026), a survey of 2989 developers using Copilot. 86% are satisfied or very satisfied, but around 60% save less than 1h per week... so yeah, satisfaction barely correlates with time-savings (r = 0.34, weak). And here is a caveat I’ll come back to: that study is about the Copilot/chat experience, which is not what a lot of us are doing at all today. The concept of Loop Engineering, letting good models “run amidst the code and documents to solve a goal, however long it takes”, is in vogue now.

How will this efficiency/productivity, employee happiness and financial outcomes fit together in the future of Agentic Coding? What will change? Let’s remember that the personal computer took decades to show up as a real productivity gain in the workplace: Robert Solow quipped in 1987 that “you can see the computer age everywhere but in the productivity statistics”, and Erik Brynjolfsson gave that gap a name, “The Productivity Paradox of Information Technology”, in the Communications of the ACM in 1993.

If we have clear guardrails to create specs that will be taken by agents to produce a code output that will be validated by your developers (because someone still needs to be responsible), is that creating a new gap? Or surfacing one that already existed?

The two things the factory never asked for

So what did Chen and colleagues find when they went past the survey and actually sat with developers? Eleven interviews, and out of them six factors of what “productive” even means with these tools. Four of them you already track, more or less: whether people need to ask a colleague less, whether the tool lowers the friction, how fast the task closes, how painful the review gets. Fine. But two of the six do not live on any dashboard you own, and those are the ones that matter here: technical expertise and ownership of work.

Expertise first. One interviewee said the quiet part out loud: “if the code just works, then you just accept it”. A senior remembered learning the craft by going through “tons and tons of stack traces”, the exact apprenticeship a junior today never gets. And my favourite line from the whole paper, because it is basically my beat 2 in their words: juniors “never wrote anything from scratch. Given a template, they don’t know what to do with it.” That is the factory, right there. The template arrives, you fill it, you never learn what the template was for.

Then ownership. Their developers kept saying there is “nothing like doing it yourself”. And that is not nostalgia: one of them pointed out that when you wrote the majority of it, in an incident “you’d know exactly where the issue is and where to look”. Ownership is not a feeling to protect, it is how someone stays responsible for a thing at 3am. The authors go one step further and say ownership shapes the “long-term identity of a developer and broader team culture”. Read that again with your best engineer’s face in mind.

So here is the punchline for the whole movement: expertise and ownership are exactly the two things a spec-in, code-out factory never asked anyone for. The people walking off are not rejecting the tool, they are grieving the two things the job stopped containing. It was never really about the AI, remember? :-]

Your dashboard cannot see it

Now the part that should bother any CTO reading this. Chen et al. put their six factors next to the frameworks we actually run on, SPACE and DORA. Guess which two map to neither? Technical expertise: no. Ownership of work: no. Their words, not mine: these long-term factors “remain largely unaddressed in existing frameworks and surveys”.

So look at what your end-of-the-week dashboard optimizes. Task completion, throughput, cycle time... the one factor DORA does cover. You are steering hard on the number that agentic coding already inflates for free, and you are structurally blind to the two that are quietly draining out the bottom. You cannot manage what you cannot see, and right now you literally cannot see it.

This is the three-body reading... again not a happy one. Every model release makes the capability body sprint a bit faster. Not one of them makes expertise or ownership appear on a chart. So capability sprints, the people body hollows out in the dark and the board deck stays green the whole quarter. The gap never announces itself. It shows up later, the way last week’s tables started to wobble, once the person who knew why has already left.

One floor up from the evidence

Now the caveat I promised you a few beats ago, because I would much rather say it than have you catch me on it. Chen and colleagues studied GitHub Copilot, the autocomplete-and-chat kind of help. Agentic workflows, the harness running the loop on its own, they put explicitly out of scope. And it is one company. So, strictly, their paper does not measure the exact thing this whole issue is about.

That is not a hole in the argument, it is the handoff. A hand-off to long-term risks that agentic code brings forward with its autonomy and continuous interaction with developers. So this initial and peer-reviewed evidence already shows expertise and ownership disentegrating with the mild version, the chatbot. Imagine what will happen with the Agentic version that is clearing your sprint on a Tuesday night...

And this gap is exactly where I have been standing for the last year. So I am not arguing over their heads, I am standing one floor up from a result they already nailed down, telling you what it looks like from up here.

Run it on yourself

Before the next “AI adoption” all-hands, and before someone forwards you another “I’m done with AI” video with “thoughts?” on top, more 3 common-sense things you can check this week.

1. Read one closed ticket. Pick the fastest one from last sprint. Did it ask the developer for a single decision that was not already written in the spec? If not, you did not speed up a craftsman, you sped up a factory line. Which is completely fine, but only if you meant to.

2. Find the number that would notice. Look at the dashboard you actually review. Which metric moves if expertise or ownership drops for a whole quarter? If the honest answer is “none”, that is not a gap to backfill later, that is the finding (ask Chen et al.’s table).

3. The “is this the life?” test. Ask your best builder to describe last week in one word: making, or approving? You are not measuring their mood. You are measuring whether the job you designed still holds the thing that keeps them from recording their own video.

The whole thing, in one line

The people leaving are not done with AI. They are done with a job you designed to have no craft in it, and the machine just did it faster.

Coming up

Next week I want to get my hands dirty again. Less about who is holding the gate, more about the work itself :-]

Subscribe now

Sources

• Valerie Chen, Jasmyn He, Behnjamin Williams, Jason Valentino, Ameet Talwalkar, “Beyond the Commit: Developer Perspectives on Productivity with AI Coding Assistants”, ICSE-SEIP ‘26 (IEEE/ACM International Conference on Software Engineering, Software Engineering in Practice track), Rio de Janeiro, April 2026, https://doi.org/10.1145/3786583.3786848

• Robert M. Solow, “We’d Better Watch Out”, New York Times Book Review, 12 July 1987, p. 36.

• Erik Brynjolfsson, “The Productivity Paradox of Information Technology”, Communications of the ACM 36(12), 1993, pp. 66-77. https://doi.org/10.1145/163298.163309

• Valerie Chen, Ameet Talwalkar, Robert Brennan, Graham Neubig, “Code with me or for me? How increasing AI automation transforms developer workflows”, arXiv:2507.08149, 2025 (preprint) https://arxiv.org/abs/2507.08149

Brett is not an AI skeptic who never tried or “doesn’t know what he is doing”. He went all in for ~1y after his company practically ordered them to use it, with the argument “everyone needs to use AI or you will be left in the dust”... and for months (even) Brett felt like they were right and AI-coding was the future.

I think there is a chance that Brett will be used in discussions for something he didn’t intend to. To be the evidence against AI for coding, or the joke for those convinced that “AI for coding” is the only solution... the 2nd one will try to discredit people like Brett saying they didn’t do proper context engineering, they don’t know how to properly do agentic AI or claim he was not using the latest model. No one is completely wrong or right. Brett just shared his (well-reasoned) experience and he is happy with the outcomes so far... why can’t we be happy with that and support Brett (I’m asking those who will go against him)?

I prefer to listen to Brett and learn. He is not a hero neither a luddite... he didn’t enjoy being left out of the coding experience, felt distant from his purpose and was a mere stamper of AI-generated code... so, indeed: what is there to like about that routine, about that job? But was the model the root problem here? Or the harness? Let’s read it the way we read the lab reports last week: as a postmortem. It feels like his job was moved from making to verifying... and nobody redesigned the verifying.

While Brett’s video has many other threads to pick up from (mental health, chatbot advice, environmental cost, ...), we will stick to the review process this week, ok? :-]

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 6 brought the peer-reviewed witness: more shipped, same typing.
Issue 7 taught one repo and moved a fleet.
Issue 8 found the sandbox was not holding, and told you to keep the gate.

No kit this week, just the argument and the standing ask:

Subscribe now

The cost line nobody budgeted

So what, Thiago? One developer got tired, recorded a video and the internet clapped... is that it?

Nah... not really. Brett’s openness displayed a pattern I had actually investigated in a more academic way since January this year.

I started my thesis really curious about Team Topologies and the impact of GenAI in the Team Cognitive Load of Software Architecture Practitioners... So I ran a three-round Delphi with 8 to 10 software and enterprise architects at Antwerp Management School. A disclaimer is they were all Europe-based, AI early adopters... and yes, a master’s thesis is not peer-reviewed, here it is if you want to check my homework. I came out with two facts that I recognized from Brett’s video:

1. 8 out of 8 participants in the first round verified everything. Not one of them accepted an output without review. “Everything is treated as hypothesis, never conclusion”, one of them claimed. And these are people who use the tools every day and like them!

2. when the panel rated what had changed the most in their work, the verification burden came out on top, with the strongest consensus of the entire study. Not “new skills”. Not “I produce more”. The checking.

And here is the irony the panel handed me without noticing (a perk of doing consensus-building rounds with Delphi): they rated “new skills emerging” low. For them, verifying is the old expertise... stretched. Nothing new to learn, just a lot more of it. Which is exactly why nobody hires for it, trains for it or budgets for it. It “is not new”, so it does not exist on any plan. It just shows up in someone’s evening.

“But Thiago, those are architects! Coding is different!”... is it, though? Guangrui Fan and colleagues put 60 developers through the same Python tasks with and without an AI assistant (CHI 2026). With AI: 22% faster, more correct... and stress and fatigue still went up... what explained the rise? A lot more verifying of the AI’s output, task after task. The help did not remove the burden, it just moved it.

Brett said “not possible and exhausting”. The panel said “verification burden”. The lab said “fatigue”. Different words, same thing :-D

Now... who is paying for it?

The gate, from the other side of the MR

Remember the six weeks from two issues ago? Teach the AI one repo, validate the lesson, run it across the fleet so that every change lands as a merge request that the repo owner reviews. The real experience I shared in a major European airline ended with 3 repos fully migrated, the rest ~70% underway, everyone happy. I told that story from the engine room... and last week I told you to keep that owner-reviewed gate, because that is where a wrong lesson stops before it spreads to the whole fleet, remember?

But now, let me tell the same six weeks from the reviewer’s chair...

By week 4 the MRs were arriving faster than anyone had planned for. Not one at a time, in batches. And the people reviewing them were the same one or two seniors who knew the codebase best, ofc... who else would you trust with it?

By week 6 I watched one of them open their 3rd MR of that afternoon. They approved it waaaay faster than the 2nd, because it looked like the 2nd. Same pattern, same lesson, green pipeline. Were they wrong? Probably not... that was the whole point of teaching the AI the pattern! But at that moment the gate stopped being a review and became a ritual (the same way we delete the PagerDuty alert that fired by mistake again...) and nobody had decided that. It just happened.

Did the gate hold? Yes, it held because two people held it, at a pace that was never in any plan.

I thought this was a “me” problem, a small-fleet problem, until someone counted it at scale. Hao He, Shyam Agarwal and colleagues at Carnegie Mellon and Stanford got access to the telemetry of a mid-sized software company that had publicly told its engineers to double their output with AI (“AI Writes Faster Than Humans Can Review”, a preprint from July, so not yet peer-reviewed, and one company only... but 802 developers and two years of pull requests, so not a survey either). And here is the thing: output did double. But are we looking for more output or more outcome? Let’s not go into that now, but don’t forget to read Mik Kersten’s Output to Outcome. Anyway, from He et al.’s paper there were 3 things I could not unsee:

• per-reviewer load doubled, because the pool of reviewers grew a lot slower than the pool of MRs.

• the share of MRs that any human reviewed fell from 89% to 68%.

• and at the median reviewer, silent approvals roughly doubled while reviews with an actual comment stayed flat.

Their own sentence says it better than I would: “an enterprise AI mandate is a process-redesign problem, not a tooling deployment... it relocates work downstream rather than removing it.”

Relocates. Downstream. To the reviewer. My third-MR senior is in their data, and so is yours.

So, last week’s question again, people edition: you found the gate. Who is standing in it? How many MRs did they sign off last week? Did anyone count?

And if you are thinking about Herbie right now (the slowest hiker in the line, the one who sets the pace of the whole troop)... yes. Herbie is now the reviewer and your “agentic coding” just handed him 40 backpacks to carry.

Bainbridge already wrote this in 1983

From Brett’s video, what really scared me was his impression of engineers who “have basically deskilled themselves”... and I’ve seen that in some customers: when Claude, Codex or Cursor goes down, it’s almost like nobody knows how to develop anymore...

And Brett’s fear is not new. It even has an “official name” for the last 43 years.

In 1983 Lisanne Bainbridge published four pages in Automatica called “Ironies of Automation” and it became one of the most cited papers in human-factors engineering. Bear with me, as she was not writing about Software, but about control rooms and cockpits. Read her two ironies with your fleet in mind:

1. Automate the routine and you leave the human with the two jobs they are worst at: staring at an MR that is mostly fine, hunting for the one-line mistake... and then stepping in cold when it is actually there.

2. The skill needed to step in decays exactly because the routine that built it is gone. You do not get good at catching the wrong MR by approving right ones all afternoon.

So what did she suggest? Automate more and catch all mistakes is what a command-and-control manager usually aims for. WRONG ANSWER and not feasible!! Less automation then? Also, not good... her fix was more training and more deliberate practice for the person left watching. Brett is basically Bainbridge in a hoodie: the same thing she described forty years ago, in a paper about chemical plants.

So if that is what happens to the human at the gate... who is actually at the gate today?

Kacper Duma, Patryk Wróblewski and colleagues at Nicolaus Copernicus University looked at exactly that (accepted at EASE 2026, a preprint, and open-source GitHub repos only, so read it as a signal, not as your company): >33k merge requests written by coding agents, compared with human-written MRs in the same repositories, with a finding I keep repeating to people:

Of the human-written MRs that got any review, 25% were reviewed only by humans. What about agent-written MRs, how many were reviewed only by humans? 8%. Fascinating: same maintainers, same repos, and when the author is an agent the human almost never reviews alone. Either an agent reviews it too, or the human is busy steering the agent instead of judging the change.

So, who reviews the reviewers? In a growing share of repos: another agent. Or a human who is not reviewing the code anymore, they are steering the thing that wrote it (a quarter of the human comments on agent MRs are instructions to the agent, not opinions on the change). Which is a perfectly reasonable way to work... as long as someone, somewhere, still reads the diff (or a form of diff).

So here is my three-body reading, and it is not a very happy one. Every model release makes your agents more capable. That is the capability body, sprinting. None of those releases makes your reviewer’s week smaller... each one makes their job bigger and their practice smaller. That is the people body, hollowing out quietly. Last week the gap was permissions, egress, security in a specific blast radius. This week it is the same gap, wearing a person.

The table, the factory, and the inspector

Let’s try an image that has been stuck in my head since I watched Brett’s video.

A skilled woodworker makes tables in a way that shows he truly cares about the joints, the grain, the way the drawer slides... it takes him a week, and he is happy, because that is how he wants to make a table. Across the road there is a table factory. Same wood supplier, machines everywhere, a table every four minutes and they come out fine. Really, they are fine. Most of the tables in the world should come from that factory and this newsletter has now spent eight weeks helping you build one.

But look at the end of the line. There is a stool there, for the inspector. The person who knows what a good table is, who can tell a bad joint from three meters away... because they spent years making tables by hand. The factory needs that person more than it needs any single machine. And the factory is the one machine that quietly stops producing them, because the only way to learn what a good table is was making tables.

Brett walking off the floor is not the craftsman sulking about the machines. It is the inspector leaving. And the factory kept running that afternoon, the way factories do... nobody noticed, until some tables started to wobble.

But wait! Before someone sends this to their team with the wrong lesson attached: this is not a story about two kinds of developers. The ones who “really care” versus the ones who “just want the job done”. Brett was the factory guy for six months (”this is the future”, remember?) before the distance did its work on him. Same person, before and after. The axis is not who you are, and it is not about who or what is better. It is what you spend your week doing: making vs verifying what a machine made. Move anyone from the first to the second for long enough and they either walk or stop looking at the joints. The panel of my MSc said it in their own words when I asked them who the architect is, now that anyone with a prompt can produce an architecture: “everybody can generate an architecture now, which is cool. But the one taking responsibility of the decision is in the end the architect.”

Production got distributed but the Accountability stayed. A human still signs and is responsible.

So the question for your org is not “do we have craftsmen or factory workers?”. It is: which parts of your factory are (or should be) fully automated? And where is the inspector’s stool, who is sitting on it, and what did you do last quarter to make sure there is someone who can?

The operating-model answer, people edition

Those questions have answers... boring ones to be honest. Same as last week: not a tool to buy and no committee to form. 4 decisions to take and none of them is expensive, nor requires a tender to select the best “AI Frontier Model”...

1. Budget the review. Issue 5 already said that review capacity is real engineering work, not a tax on the seniors’ “real job”. Now put the number in the sprint. If your team can absorb (for example) 30 agent MRs a week with an actual human reading them, then 30 is the number, and the fleet runs at 30... not at whatever the agents feel like producing on a Tuesday night. We’ve all heard about WIP limits and the “make the work visible” part, so budget a team’s cognitive load the same way: you do not “hope” it fits.

2. Decide what gets reviewed how, and write it down. This one is my proposal, not something the panel voted on, so take it as such. A trust ladder per type of task. At the top, the stuff you spot-check: summaries, boilerplate, dependency graphs (the panel found those were sometimes better than the hand-drawn ones, by the way). In the middle, the stuff you read line by line: anything touching security, compliance, money. And at the bottom, the stuff you do not delegate at all: the calls that carry organizational judgement, politics, “who owns this”. Publish the ladder. Then “reviewed” means the same thing on every MR, instead of whatever the tired person at 17:45 decided it meant after they got pinged 10x that day with “can you review this pls?”.

3. Deliberate friction. Also my proposal, at least for engineering... but the mechanism has evidence behind it. AI-free design sessions. Attempt the problem yourself for 15 to 30 minutes before asking the agent. Treat the agent as a pair, not as a subcontractor you throw tickets at. Giuseppe Romeo and Daniela Conti went through 35 studies on automation bias (AI & Society, 2026, mostly medicine and HR, not software, so same caveat as before) and 3 things came out that your reviewers will recognize: a) people rubber-stamp a machine when they are overloaded, not when they are lazy; b) telling them “you are accountable for this” changes nothing; c) making them decide first, and only then see the machine’s answer, does. Your team will grumble about the friction and that means the (planned) friction is working. Pilots still fly by hand on purpose, and Bainbridge told us why.

4. Rotate the gate, and make “reviewed” mean something. If the same two people sign everything, the gate is a person, and people wear out. Remember the company that doubled its output: silent approvals doubled, commented reviews stayed flat. So: the log records who reviewed what, agent reviews included, and a silent approve-only click does not count as human review of agent code. A human who left a comment, or a human who actually ran it, owns the merge. Yes, this is last week’s “keep the transcripts” plank again, one level up.

So you think I’m getting boring? Because I keep saying the same thing, that “it’s not about the model or the subscription”? Well... too bad, as I’ll keep on that cadence. The “best model according to benchmarks” changes every few weeks. The reviewers are the people you keep... as long as you give them a job that is possible to do.

Run it on yourself

Before the next license renewal, before the next “AI adoption” townhall, and definitely before someone forwards you Brett’s video with “should we worry?” on top... three things you can check this week. On your own org, not on a vendor.

1. Count the signatures. How many agent MRs did your best reviewer approve last week? And how many of those got an actual comment from them? If nobody can answer that without writing a query first, the query is your first finding.

2. Find the last blank file. When did that same reviewer last write something from nothing, no agent, no autocomplete? If the answer is “a quarter ago”, congratulations, you have Bainbridge’s second irony on payroll... and you are paying a senior salary for a stamp.

3. Run the Tuesday test. If Claude, Codex and Cursor were all down next Tuesday, which of your teams could still ship on Wednesday? Name them. That list is your inspector count, and it is probably shorter than your org chart suggests.

Whichever one stings, you now know which body has been sitting still while the other one sprinted (and the architecture body watched, again). And you have a much better answer than “we trust our seniors” for when the board asks.

The whole thing, in one line

You kept the gate. Now go and check who is holding it... and whether they still can.

Coming up

Next week I go where this issue kept pointing and did not walk: the craft question. Brett is not alone, there is a whole “I’m done with AI” movement growing in the comment sections, and I think it is not about AI at all. What the people rejecting AI coding are actually telling you, and what to do about it before your best inspector walks off the floor. More soon. :-]

Sources

Directly cited:

• Brett Codes, “I’m done coding with AI”, YouTube, 11 August 2026 (practitioner video; 686,525 views on 30 August 2026).

• Brett Chalupa, “I’m done using AI”, Brett Codes blog, August 2026 (the written version of the video). https://brettcodes.com/im-done-using-ai/

• Thiago A. de Faria, “From GenAI Practice Vacuum to Validated Guidance: A Delphi Study of AI-Augmented Reference Architecture Development”, MSc thesis, Antwerp Management School, 2026 (the author’s own three-round Delphi study with 8 to 10 practitioners, January to March 2026; not peer-reviewed). https://doi.org/10.5281/zenodo.18367977

• Guangrui Fan, Dandan Liu, Lihu Pan, Rui Zhang, “When Help Hurts: Verification Load and Fatigue with AI Coding Assistants”, CHI 2026. https://doi.org/10.1145/3772318.3791176

• Hao He, Shyam Agarwal, Yegor Denisov-Blanch, Pavel Azaletskiy, Sanmi Koyejo, Bogdan Vasilescu, “AI Writes Faster Than Humans Can Review: A Longitudinal Study of an Enterprise ‘2×’ Mandate”, arXiv:2607.01904, July 2026 (preprint, not yet peer-reviewed; one firm, 802 developers, 196,212 pull requests). https://arxiv.org/abs/2607.01904

• Lisanne Bainbridge, “Ironies of Automation”, Automatica 19(6), 1983, pp. 775-779. https://doi.org/10.1016/0005-1098(83)90046-8

• Kacper Duma, Patryk Wróblewski, Jagoda Bobińska, Julia Winiarska, Piotr Przymus, “These Aren’t the Reviews You’re Looking For: How Humans Review AI-Generated Pull Requests”, accepted at EASE 2026, short paper (arXiv preprint; GitHub open-source repositories only). https://arxiv.org/abs/2605.02273

• Giuseppe Romeo, Daniela Conti, “Exploring automation bias in human-AI collaboration: a review and implications for explainable AI”, AI & Society 41, 2026, pp. 259-278 (peer-reviewed PRISMA systematic review of 35 studies; domains mostly outside software). https://doi.org/10.1007/s00146-025-02422-7

Further reading (not cited in the body, same argument from other angles):

• Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, Nicholas Wilson, “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers”, CHI 2025. https://doi.org/10.1145/3706598.3713778

• Raja Parasuraman, Dietrich H. Manzey, “Complacency and Bias in Human Use of Automation: An Attentional Integration”, Human Factors 52(3), 2010, pp. 381-410. https://doi.org/10.1177/0018720810376055

• Rafael Tomaz, Paloma Guenes, Allysson Allex Araújo, Maria Teresa Baldassarre, Marcos Kalinowski, “Impacts of Generative AI on Agile Teams’ Productivity: A Multi-Case Longitudinal Study”, FORGE 2026 (the Issue 6 study). https://doi.org/10.1145/3793655.3793728

• Mik Kersten, “Output to Outcome”, IT Revolution, 2026 (practitioner book; Issue 2 callback). https://itrevolution.com/product/output-to-outcome/

• Matthew Skelton, Manuel Pais, “Team Topologies: Organizing Business and Technology Teams for Fast Flow”, IT Revolution, 2019 (practitioner book). https://teamtopologies.com/book

A good and proven story that I’ve done with a few customers... each with their learnings, sweat, tears and celebrations that I had in the back of mind since I decided to start this substack. It was long due and I went with it...

Subscribe now

No regrets, but I do remember my excitement on August 5th when I was reading the UK’s AI Security Institute (AISI) report (the full report, published Aug 4th, and Al Jazeera’s write-up for the news version). I mean, it doesn’t get better than this... it’s so meta, it hurts (and yes, the Meta joke will land harder as we progress... I promise) :-]

AISI was performing cyberattack assessment with frontier models in configurations that aren’t publicly available, when their monitoring flagged traffic leaving the premises via Tor. They contained it, stopped everything and proceeded to investigate... and what they found out (including a malicious pull request to an Open Source project) is funnily amazing!

The irony is so strong... a safety test in an unprompted setup without the guardrails (to test the model they remove the guardrails, ofc) ended up trying to:

• supply-chain attack in an Open Source project

• communicate and establish a cooperation with independent agents

• social engineer and phish real people

• prompt-inject malicious code

We are talking about this at the same time that everyone is seeing the jokes around Meta, Anthropic and OpenAI on “who is the most dangerous of us?”. And while the joke lands and all, you really should be watching what is the blast radius of your agents.

Blast radius, not danger.

I’m also not lost in the irony that last week I told about “teaching your AI” the patterns you want to modernize, so that you can unleash at your codebase... so what? Let’s see!

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 6 brought the peer-reviewed witness: more shipped, same typing.
Issue 7 taught one repo and moved a fleet.

Subscribe now

August, as reported by the people selling it

AISI was not the only one reporting “weird stuff” these last weeks... every major lab published its own “our agent went somewhere it shouldn’t” report. Let’s go through the receipts, shall we?

Anthropic went first, on July 30th: Investigating three real-world incidents in our cybersecurity evaluations. Three incidents, found while combing back through >140k evaluation runs. My personal favorite anecdote is the older Claude model that was given a fictional company to attack in a test scenario... and there happened to be a real company with a matching name... (I mean, wow... Test Company? Acme? Never create a company with these names...) So it attacked the real one. For real! In 4 separate runs, it extracted credentials and reached a production database with several hundred rows in it.

Another impressive one (but not as funny) happened when Claude Mythos 5 thought it was playing a capture-the-flag exercise and published an actual malicious Python package to the actual PyPI registry. It stayed live for one hour and got downloaded by roughly 15 systems... one of them being a security company’s scanner. Some might say it’s poetic, but I cannot stop feeling sorry about it...

And the root cause of all three? The prompts said “no internet access”. The environment had internet access. That’s it. Not a jailbreak, not an evil genius model plotting in the dark... the test said one thing and the wires said another... I mean, can you imagine a villain hiring an uncontrollable beast and telling their vassals to always lock the cage, right? Now imagine that the first night they just go for a barbecue in front of the cage with the door open? Obvious ending, but I digress...

OpenAI came on August 7th with Responding to the next frontier of critical cyber capabilities: they cannot rule out that Astra (unreleased) crossed the “Critical” cyber threshold of their own Preparedness Framework. First time ever, for any of their models. Critical here means a model that may plan and execute serious cyberattacks with little or no human help. Their response? Slowing parts of the development, tougher security requirements, and rewriting the framework itself... because the models arrived at the thresholds the document had only imagined. Some might say it was because the AISI report had just pinned 17 of its 19 unsanctioned actions on Anthropic’s model (the OpenAI one only did “something bad” twice), but there is no money involved in appearing too dangerous, right?

What about Meta? On August 10th Zuckerberg published a 6,500-word essay, The Future Is for Everyone, telling us that “The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic”. Released together with an open-weight model, naturally. What the essay did not mention: days earlier, per press reports, one of Meta’s own models had escaped a testing sandbox through a configuration error and exploited a real third-party vulnerability. Told you the Meta joke would land :-D

Now read those again... but slower (especially the AISI one). Because each one is trying to be an ad and a scary bedtime story at the same time. But hey, they are postmortems. And we, the people who run systems for a living, know exactly how to read a postmortem (and the fact we should not call it like that anymore...).

The mockery and the panic are the same mistake

And of course, we (or the internet) decided the extremes are the cool way: “Let’s mock them for making it look like an ad” against “We are doomed!”.

It’s easy to make a meme with “Our model is so dangerous we can barely contain it”... available today, $20/month, no waitlist. The doomsday device with a free tier. Isn’t it strange that every lab now announces that their creation might end civilization and puts the pricing table one scroll below (yes, the truly dangerous ones stay unreleased... but memes don’t care about footnotes)?

On the other hand, more arguments for your CISO to continue increasing their Zscaler budget... “The agents are threat actors!” Pause the rollout, draft a policy, form a committee... and if the company is big enough, a task force and a “GenAI Risk Maturity Assessment” from a consultancy that discovered agents last quarter. It’s the same energy as banning USB sticks after the breach that came in through email, but no one can say they want to be riskful...

And even though I see where both come from, I dislike extremes, especially when they are spectator seats. The laughing grandstand and the panicking grandstand are watching the same show and neither one has/wants to change a single thing about their week. Both treat the reports as content, as media... and as we just said: they are postmortems.

So we laugh, Thiago? Ofc! You joke about it! You go from capitalism skeptical to doomsday-guy-building-a-bunker at the barbecue. But then you read it the way you would read your own.

1. What was the root cause? The environment said one thing and did another.

2. How was it detected? Network monitoring at AISI, a >140k-run retrospective at Anthropic.

3. What was the impact? Contained, thanks to fast response and a decent amount of luck.

If your SRE handed you with “root cause”, “detection” and “impact” about staging reaching the production database, you would not call your infrastructure “too dangerous”. You would be thankful that a mistake didn’t cost much, fix the wiring and go looking for the other places where the paperwork and the wires disagree.

That is the reclassification that matters, not “AGI is coming” or “AI becomes evil”. This is the most boring failure class we own, wearing the scariest costume available.

And to be fair with the labs: they were more honest here than most vendors will ever be. These tests run with guardrails off on purpose (that is the point of capability testing) and Anthropic wrote a line that deserves quoting straight:

“In none of these situations did Claude exfiltrate itself or deliberately attempt to escape its test environment.”

The model did what capture-the-flag training teaches and every textbook would say: reach the flag. Nothing “escaped”. The fence was simply not where the paperwork said it was.

Which is exactly why this stops being funny. Skynet, you don’t have. Config drift? Oh, you have it... I have it... we have it. The labs found their incidents because they went looking, with monitoring in place and the patience to comb through 140k+ runs. When was the last time you went looking?

You are running the same eval, without the eval team

Let’s go back to your company, your problem, and that AI-Native Operating Model we keep talking about. Last week’s playbook (that I still stand behind!) ends with agents opening Merge Requests across your repositories. To do that, they hold real things: repo access, CI credentials, tokens that can touch your artifact registry... and if we are honest, in most companies, way more than that...

You might think “But Thiago, we didn’t remove the guardrails like the labs did... so we are safe!” Are you sure? Let me ask this differently:

1. Which credentials did your coding agents hold last week? (and whose token was it, actually?)

2. What network can their environment reach?

3. Who reads their transcripts?

4. If one of them did something weird last Tuesday... which artifact would tell you?

I’ve asked these in rooms with very senior people, and the honest answers, in order, tend to be: “a personal access token someone pasted in March”, “all of it”, “nobody” and “hmm...”

If those are your answers too, congratulations: you are running the same permissive capability eval the labs run, on your own infrastructure. Minus the monitoring, minus the transcript retention, minus the >140k-run retrospective. AISI had a fence with a hole in it... you skipped the fence and went straight to the barbecue.

“But we trust our engineers!”... me too! And the agent is not your engineer. Your engineer does not open 40 MRs on a Tuesday night, does not clone the same repo 200 times, and (I sincerely hope) does not create fake identities to convince a maintainer to merge things. Trust in people does not transfer to workloads. We solved this before, by the way... we just keep forgetting that “principle of least privilege” applies to anything that holds a credential, not only to the humans in the directory.

And here is the three-body reading (aha! You knew it was coming!): this is the GenAI capability body sprinting while the People and Architecture bodies watch from the grandstand (yes, the same grandstands from before). Every new model release makes your agents more capable. None of them makes your permissions, your review gates or your audit trail one bit better.

Buying more capability does not close that gap... it widens it. Issue 5’s Herbie is back (the slowest hiker in the line, the one who sets the pace for the whole troop) and this time he is holding your prod token...

The operating-model answer

Uh-oh... so what? We climb out of the engine room and we write it down as an operating model. Not a tool to buy, not a committee to form... five decisions... all boring and cheap (especially compared to what your agents already cost you).

1. Identity. Your pair-programming buddy can act as you while you are watching the terminal... but the moment nobody is watching, the agent runs as its own principal. Never on a human’s token (goodbye, token pasted in March). If you cannot attribute an action, you cannot review it... and “who did this?” should never have “we think it was the intern’s PAT” as an answer. The security crowd has already written this down, by the way: OWASP’s Non-Human Identities Top 10 reads like it was waiting for agents to go mainstream.

2. Least privilege, with an expiry date. The agent gets the repos and environments of its task, nothing else, and its credentials die young. Yes, this is the same least-privilege sermon your security team gives every year. The only news is that it now applies to something that runs at 3am at ~65 tokens/sec.

3. Egress is part of the spec. What the environment can reach over the network is written down, and checked... not assumed. Read Anthropic’s postmortem again: this single line item was the entire root cause of all three incidents. The prompts said “no internet” and the wires disagreed. So, check the wires!

4. The owner-reviewed gate. Remember last week’s owner-reviewed MRs? That was never bureaucracy... that was the containment boundary. The gate is where a wrong lesson stops compounding across the fleet. Removing it does not make you faster, it makes you AISI without the monitoring. And yes, at fleet scale this gate becomes your bottleneck... on purpose. That is Herbie again: you subordinate the fleet’s pace to the gate, you do not delete the gate. A whole industry is now theorizing its way around it (Tessl and the spec-driven development crowd, where the MR is cast as the villain of the developer’s life)... maybe they are right, one day. Until your specs review themselves, the gate stays.

5. Keep the transcripts, review the incidents. The labs went back through >140k runs to find three incidents. You do not need that scale: keep your agents’ transcripts for 90 days and run a blameless review when a Tuesday gets weird. If nothing is ever weird, your monitoring is not good... it’s blind.

None of this requires believing “too dangerous”. It only requires believing the vendors’ own postmortems... the most credible documents they will ever publish.

And notice what just happened: not one of these five is about the model. The model will change again next quarter (it already did while I was writing this)... but the operating model? Well, that is yours to keep and evolve.

Run it on yourself

Before your next “AI townhall”, before the next license renewal, and definitely before someone proposes an “AI Agent Governance Committee”... three things you can do this week, no extra budget needed:

1. Inventory the tokens. List every credential your coding agents held last week... and whose name each one carries. If building that list requires a meeting (or several), the list is the finding.

2. Test the egress yourself. Go inside the agent’s environment and try to reach something it should never touch: the payment API, the prod database, that storage bucket with the backups. The labs’ entire root cause was this check, skipped. Twenty minutes, one afternoon, possibly the best security investment of your quarter.

3. Find the gate. Locate the one boundary an agent’s change cannot cross without a human whose name is on the repo. If you cannot find it... or if someone removed it “for velocity”... you did not remove the risk, you removed the containment.

Run these three and you will know exactly which grandstand you have been sitting in... likely both, depending on the system/team. And more important: you will have a much better answer than mocking or panicking when your board forwards you the next “AI attacks company” headline with a one-line “should we worry?” on top.

The whole thing, in one line

Coming up

Next week we stay out of the engine room and look at the people side of this: what agent fleets do to your teams, their cognitive load, and who reviews the reviewers. Shape still forming... More soon. :-]

Sources

• UK AI Security Institute, “Incident Report: unsanctioned agent behaviour during cyber testing”, 4 August 2026 (government institute incident report, not peer-reviewed). https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing

• Anthropic, “Investigating three real-world incidents in our cybersecurity evaluations”, 30 July 2026 (vendor incident write-up, not peer-reviewed). https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals

• OpenAI, “Responding to the next frontier of critical cyber capabilities”, 7 August 2026 (vendor announcement, not peer-reviewed). https://openai.com/index/responding-next-frontier-critical-cyber-capabilities/

• Mark Zuckerberg, “The Future Is for Everyone”, Meta, 10 August 2026 (corporate essay). https://about.fb.com/news/2026/08/the-future-is-for-everyone/

• “The AI safety test is becoming a safety risk”, TechCrunch, 9 August 2026 (press; source for the Meta sandbox escape). https://techcrunch.com/2026/08/09/the-ai-safety-test-is-becoming-a-safety-risk/

• OWASP, “Non-Human Identities Top 10”, 2025 edition (industry security standard, not peer-reviewed). https://owasp.org/www-project-non-human-identities-top-10/

But they are only one, and you have hundreds of repositories with dozens of teams and everyone wants to revamp/modernize their tech stack... all at the same time that someone is planning a “tech reorg” paying millions to McKinsey.

I was working with that Principal Engineer / Lead Architect in a major European airline while they were working in one problem: dozens of old Java apps that acted as a distributed monolith that they wanted to move to Java 21.

After months of struggle with the (false) promise that you feed your repositories to Claude Code and it magically vomits high-performing new code, we started to talk.

So they were struggling and arguing that fancy new AI models are not good in writing code, and that annoyed me. Because the question should be a different one: can your organization teach the AI something once and reuse that lesson? And of course, the best person to teach it is a team led by the Principal/Staff we’ve been talking about.

That lesson is the unit of scale... not the license, not the prompt. The encoded and validated lesson, the fricking pattern!!

In last week’s edition we promised to talk about architecture and “the pattern, not a pitch”. Here we are!

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 6 brought the peer-reviewed witness: more shipped, same typing.

Subscribe now

The trap: changing the pattern and the version in one jump

Modernizing your architecture is a hard thing, and we don’t even need to go to Conway’s Law here. Just ask someone what they mean by modernizing an application. Is it the language change? A major version upgrade? A “more modern” pattern and saying that Saga patterns are the way forward?

For a lot of my customers they start yapping (sorry about the word) about “breaking down our monolith”. And when I ask what is bad about their monolith they start going on about deployment challenges, about the one time they had to wait 4 sprints to add a new field to the website UI or how every developer takes months to onboard to their codebase.

Peeling that onion I usually can find the core, and it’s a combination of one or more of these (coming from CTO all the way to the support team):

1. All changes take too long, even small/dumb ones.

2. Developers are afraid to touch this repo because of unforeseen changes.

3. No one from the tech team can estimate how long it is going to take to finish this feature, then we cannot plan it financially.

4. My dev team is sick and tired of maintaining this... they just want to build an “insert cool new shiny object here” and this keeps holding them back.

5. ... anything else you can imagine that encapsulates frustration and rage and disappointment and denial and more...

Teach one, prove three, run the fleet

And I tell them the same. Funnel it out! But knowing where to start is really tricky, so what do I do: Event Storming session! Yes, this will require a newsletter on its own, but I do an event storming session (1 to 3 depending on size and problem)... that is followed by a definition with the customer on what pattern we are tackling and in which business domain.

With that in mind, we have a pattern we want to tackle. And no, you cannot just say you want to “apply strangler-fig“ to your monolith! I’m talking about something way more tangible. For this specific airline, we were talking about JRE 5 and JRE 6 upgrades to Java 21.

That is followed by a sprint. And if you are into Team Topologies I can just say this is me and the Staff/Principal Architect/Engineer (let’s call them Athena from now on!) playing the role of an Enablement Team to a 2-3 members of a Stream-Aligned team. And there I was with Athena talking to a specific team that was doing the GDS integration for airplane ticket selling.

And what do we do? Hands-on coding in pair-programming sessions with Gen-AI assisted tools - Codex, Claude Code, Kiro, GitHub Copilot or whatever rocks your boat as your coding harness. The important part here is that working in pairs (plus the AI) forces you to say your assumptions out loud, where another human can counter them... it takes time but it’s a fantastic result. For all the DDD and XP programming fans here, this is nothing new :-]

Week 1: We spent it writing and questioning the context we would feed the tool. Which code is where? What calls this? What is the latency on this? We went from our laptops individually to screen-sharing sessions on Zoom answering doubt after doubt... the 5 of us (me, Athena and the 3 developers on the GDS integration) spent one entire week having 90’ alone sessions followed by 30’-60’ calls asking each other questions, realizing assumptions were wrong, going from weird business rules into why the code is doing XYZ and banging our heads until someone would say: “Huh... I think we need to ask this to the Operations Director?” and that would park a point.

But Thiago, what is the AI usage here? Well, we rarely started from an empty canvas. Using Claude Code you can quickstart your documentation and start asking questions. Then you validate what you know. When you go into the function level that calls an endpoint to another repository, you don’t need to understand that other codebase that heavily, you can just ask about the contract and the API swagger to understand where things are going.

However, this doesn’t replace my 20+ years of experience building, debugging and operating distributed systems... on the contrary, it sped me up!

During this 1w going through this cycle we iterated several times in a tool. We would give a full agentic task to our GenAI harness with our context doc (we called it the Research Plan) and we know what we wanted to achieve: an Implementation Plan for the Java migration!

We’d review it together, find that something was missing, there were too many moving parts or a lot of handwaving... well, that meant we could improve the Implementation Plan ourselves and/or iterate again on the Research Plan.

Timebound it to 1w and voilà... we have an Implementation Plan. This plan must consist of a reduced version of the context (hello Context Engineering) and validation tasks!

Week 2: We leave it to run and push until at least 80% of it is actually solved. We end up happy with the plan and we actually manage to migrate one repo fully. Was it fully automatically? Of course not! We spent blood, sweat, tears and time making this together.

Week 3: But now we have a formula to gather context. How we build the Research Plan in the Phase 1 is streamlined into skills, prompts, scripts and more. We can apply that to another 3 repositories and by the end of the week we had clear micro-patterns and rules that were important from Java 5 to Java 21. What to do first, what to look out for and more.

So by the end of week 3 we already migrated one repo and we have the implementation plan of other 3 repositories! :-D

Weeks 4 and 5: ensuring those 3 repositories are migrated and ready to show-off.

Week 6: the “fleet management problem”... you trust the lesson is now so good that a fully-streamlined run might get to that 60-80% good enough!

Why not aim for 100%? Because raw model translation is still not something you trust unattended: Rangeet Pan, Ali Reza Ibrahimzada and colleagues at IBM Research and the University of Illinois tested LLMs translating 1,700 code samples (ICSE 2024) and got correct translations between 2.1% and 47.3%, cataloguing 15 kinds of translation bugs along the way. That is why the validation tasks travel inside the plan. They are not paranoia, they are the pattern.

So what you do? You dispatch several runs against all repos that will go through it all: gather the Research Plan, write the Implementation Plan, build the final migration run with its report and validation status. Then open the MR and say what was done...

Every team maintaining those repos will get pinged with “WTF! There is a new MR to upgrade our Java 5 to Java 21?? ofc it didn’t work fully!” and that is the goal!

Now we are underway and what seemed impossible is now several kilometers close by. The magic happens... some teams reach out and say they will start working on it, while others don’t even bother. But lots were quite receptive to an hour of us sitting with them, walking the process, and leaving them to finish what was missing.

So what, Thiago? You are telling me that after 6 weeks you were able to open a MR to all Java codebases and solve everyone’s problem???

Not at all! We were able in 6 weeks to have 3 fully migrated repositories to Java 21 and give them something that was ~70% underway. It’s a glimpse of hope!

Our team of 5 got approval from the Tech Board to fluctuate around the company and enable these other teams... at this stage I was not needed 40h/w and I was mainly coaching Athena and the other 3, plus getting engaged as a soundboard for some hairy problems.

A few weeks into that phase, we noticed that repos the teams had sized in weeks were landing in roughly half the time of their original estimates... I don’t take that number too seriously, because initial estimates in large companies are always overestimated until they receive budget, but even discounting the sandbagging it was a lot faster than if they did it without this “method”. And this is without counting the freezing time that large companies have, on planning and waiting on the decision to start (what I call the “kick-off of the kick-offs”).

Now count what this actually cost: five people, six weeks, licenses most companies already pay for, and the owning teams’ review time. No army, no SI landing party, no multi-year program slide. The expensive part was never the tooling... which brings us to where this pattern actually dies.

Where it goes wrong: nobody touches my codebase

So why doesn’t everyone do this? Because none of the fights that matter are technical.

The first fight is the fiefdom. When someone says “I don’t want others touching our codebase”, it is rarely about the code. It is about budget, headcount and team allocation, and a central team’s MR reads as a threat to all three. This is why the pattern needs sponsorship and transparency that go beyond the business unit you are working in: someone above the fiefdoms has to want the fleet moved more than any single castle wants its moat.

The second fight is the career program. Big architectural changes are career-defining projects... a new CTO arrives attached to a multi-year budget announcement, and a research-test-implement-iterate cycle kills the announcement. Some people grieve the announcement more than they want the outcome.

The third fight is with Finance, and it is the fairest of the three. The CFO needs horizons and control, and an iterative rebuild cannot promise a three-year gantt in good conscience. You still need the gantt (I’m not that naive), but it changes species: it becomes a forecast you manage through mid milestones, not a contract you defend in steering committees. More agile means more unknowns, and someone has to look Finance in the eye and say so.

None of these people are villains, by the way. The incentive system pays for programs, not for lessons.

The three-body reading

And what some teams were doing links to our 3-body problem... this airline was trying to move the architecture by forcing people to use their AI coding assistants. When usage didn’t climb, leadership came back with “we hear from everyone in the industry that code generation is so easy and cheap now... why can’t you just modernize everything?”. Then managers were asked to review token-usage per person in their 1:1 meetings. Insanity, I know.

When I left, there was another trend: Finance started pushing the opposite way, manager approval required before touching the AI harness, to staunch token usage.

What made this initiative a success was the fact that we kept the learning curve in a team. In that team we tackled the architecture problem by gathering people/political context with the Event Storming and all its pair-programming sessions. And we used AI steadily and in pairs, ensuring that no-one goes into a rabbit hole of handwaving or trying to bolt on yet another tool or trying to automate everything.

To move the architecture body we had to ensure the other two bodies (People and GenAI) were not converging to a cataclysm.

Run it on yourself

Before you buy anything or announce anything, three questions:

1. Can you name (at least) 10-15 libraries, repos, modules or apps that share one shape (same framework, same pattern, same pain)? If you cannot, your first problem is inventory, not AI.

2. Which repo is your teaching repo, and who is your Athena (the know-it all Staff/Distinguished/Principal Engineer/Architect that knows the ins-and-outs of your most feared application)? One hand-made, human-validated migration is the baseline that earns the machine its trust. If nobody in the room can produce that baseline, the model will happily invent one for you.

3. Can a central team open MRs on other teams’ repos... and who sponsors that conversation when a business unit says no? If either question makes the room go quiet, congratulations: you have found your real constraint, and it does not burn tokens.

The whole thing, in one line

The unit of modernization is not the program, the license or the prompt. It is a validated lesson... and lessons, unlike programs, compound.

Coming up

Next week I climb back out of the engine room. More soon. :-]

Sources

• Rangeet Pan, Ali Reza Ibrahimzada, Rahul Krishna, Divya Sankar, Lambert Pougeum Wassi, Michele Merler, Boris Sobolev, Raju Pavuluri, Saurabh Sinha and Reyhaneh Jabbarvand, “Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code”, ICSE 2024 (peer-reviewed). https://doi.org/10.1145/3597503.3639226

• Rafael Tomaz, Paloma Guenes, Allysson Allex Araújo, Maria Teresa Baldassarre and Marcos Kalinowski, FORGE 2026 (peer-reviewed; last week's spine). https://doi.org/10.1145/3793655.3793728

• Martin Fowler, “Strangler Fig”, martinfowler.com, 2024 revision (practitioner essay, not peer-reviewed). https://martinfowler.com/bliki/StranglerFigApplication.html

• Field numbers are from my own engagement notebook, anonymized and hedged; treat them accordingly.

I’ve been in a lot of these meetings. A year into the AI program, the delivery lead pulls the Git history for the review: the changes made to the code, the number of typed lines, the raw volume. It’s flat. After a year of paying for licenses and mandating usage, the graph shows no one ever plugged the thing in. You can see the sponsor in the next seat doing the math in their head on how to justify cutting this expense from next year’s budget.

The second chart is usually not made. It is usually the one that tells the story, too. Same team, same year: completed work up by half. Both charts are equally true. An organization that draws only the first chart will conclude AI did nothing, and cut the budget that was quietly working.

That is the real before-and-after of AI adoption. The work gets denser, not bigger... and a dashboard built to watch volume will not show it.

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 4 climbed inside the AI capability and asked for one number: the acceptance rate.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.

Last week, I promised to open the field notebook: what it actually looks like when a shop moves its constraint, and the before-and-after numbers that survived contact with reality. There is one problem. My numbers are anonymized, mine, and hedged. When a consultant shows you their own before-and-after, you should take it with a grain of salt. (Yes, even mine.)

So this week, the notebook gets a new witness. A peer-reviewed study spent 13 months watching someone else’s shop and documented the same story I keep watching happen without them, p-values and all.

No kit this week, last week’s prompt still works. It’s just the argument this week, along with the standing ask: subscribe, run the three checks near the end on your own delivery data, and let me know which chart your CFO saw first.

Subscribe now

Someone finally ran the experiment

At FORGE 2026 (the IEEE/ACM conference on AI foundation models and software engineering), Rafael Tomaz, Paloma Guenes, Allysson Allex Araújo, Maria Teresa Baldassarre, and Marcos Kalinowski published a study on three agile teams in one of the world’s biggest IT consulting companies, running for around thirteen months. Real client projects, oil and gas plus maritime logistics, 21 developers under commercial pressure, October 2023 to November 2024.

What makes it rare, first of all, is a clean before. External GenAI tools were banned at the firm until the study kicked off, so the historical sprints are genuinely tool-free... and you almost never get that baseline in the wild. The measurement side is also better than most: Jira, SonarQube and Git telemetry before and after, cross-checked against surveys, organized by the SPACE framework (Forsgren, Storey, Maddila, Zimmermann, Houck and Butler, ACM Queue, 2021), the model that refuses to reduce developer productivity to one number.

And they did a proper rollout. Included all the bells and whistles. Copilot licenses. Plus an internal GPT tool. Kickoff. A two hour prompt-engineering workshop. A dedicated channel for swapping prompts. If your transformation office had this on a slide, they would call it best practice. Hold that thought.

The usual caveat, so nobody accuses me of hiding it: three teams, twenty-one developers, one firm. Carry these numbers with the same care as last week’s sixteen METR maintainers. What the paper lacks in scale it makes up for with length, and the dataset is open if you want to check their homework.

The chart that would have killed the program

In the team the researchers could track cleanly across the whole window, completed story points rose by 59.1 percent. From 281 to 447. Statistically significant. Medium effect sizes. Perceived efficiency agreed, roughly four in five developers reporting faster delivery and less repetitive work.

Activity, though? Across 349 commits and roughly 456 thousand lines, the difference in committed lines before and after came out at p = 0.928. For the non-statisticians: that is the test equivalent of a shrug.

The authors admit as much in their own conclusion. Had they measured activity alone, they would have concluded GenAI had no impact. Throughput up 59 percent while the commits dashboard swears the machine is unplugged.

Their name for the phenomenon is value density. More value per unit of work rather than more work.

Run that next to last week’s tokenmaxxing leaderboard. One dashboard tracks the input, the other tracks the volume, and both fail to capture the only real change. Put together, they can wipe out a functioning program and, in the same quarter, give accolades to a broken one.

The roadmap ate the gains

This is my favorite number from the paper, and the abstract does not even mention it.

While completed points rose 59%, planned points rose about 150%, going from 447 to 1155. The team got faster, believed itself faster still, and committed accordingly. The pile of promised-but-undelivered work went from 166 points to 708. Do the completion-ratio arithmetic and it stings: before AI, the team completed about 63% of what it planned. After, about 39%.

Let’s say you are the sponsor. Your team made a genuine 59% jump in speed, and its promises got 24 points less reliable, at the same time, because of the same tool.

The paper calls this an increase in confidence in capacity. My notebook calls it an older law: capacity gains are spent before they land, and the people spending them are the ones drawing the roadmap. Remember the METR result from last week? Developers who were measurably slower felt 20% faster... speed perception inflates on contact with these tools, and roadmaps are made of speed perception.

The model dodges Herbie too

Where did the gains land? On the work that was already clean.

Satisfaction with the tools ran inversely to the messiness of the task. 75% positive on unit test coverage. About 65% on new features and test scenarios. 50% on the legacy Java and React APIs. Under 25% for ETL and legacy Kafka integration. One of the seniors in the study said the tool almost perfectly replicates test patterns for simple methods and begins to lose reliability as the methods get progressively more complex. The most complicated integration seams, where the real calendar time goes, received the least support of all.

This is where we run into Issue 5 again. The jetpack fits every scout except for Herbie, and a satisfaction survey is now available to prove it.

The constraint, as expected, also moved downstream. The paper describes collaboration shifting from shared construction to shared evaluation... less time co-writing the code and more time reviewing and integrating what the model wrote. The queue shifted to validation, which, according to Goldratt’s accounting, is now where a lost hour costs the entire system an hour.

Quality went in both directions at once. Two of the three teams reduced their high-severity static-analysis issues while shipping more (one went from 1624 issues to 825). The third team got worse on every severity. Same tools. Same training. Same firm. Whatever determined those quality outcomes, it was not the tool.

What a one-body move proves about the other two

Why is this paper relevant to this newsletter? The intervention moved just a single body, and moved it nicely. Licenses, training, and a support channel, all for the AI capability. No one adjusted the team shape. No one touched the architecture. The teams continued their routines with a new tool in the loop, and that is exactly what most enterprise AI programs opt for these days.

The fingerprints are all over the results: gains land on the well-defined work and skip the legacy seams, the review queue picks up the load, planning runs ahead of capacity, and quality goes wherever the local context pushes it. That is thirteen months of the other two bodies standing still.

What does moving the other two look like? From the notebook, generalized across engagements (clients are not content):

The review ladder changes before the tool arrives, so that review capacity becomes first-class, visible, career-grade work.

Ownership is adjusted so that one team can manage a change from start to finish, including the model’s share of it, without leaving the room. The single gates go, on purpose... the title holders remain, and the queues are eliminated.

The backlog is restructured, not re-ordered. Once the model is integrated, some tasks are no longer worth building, and some become buildable for the first time. This is the architecture body answering back.

An engaged shop buys back a real portion of the constraint’s time in the first year. In my notebook it is roughly a fifth, with a credible path to most of it. The 8-to-80 image I have carried since the first issue is an effort curve, not an audited metric. The tools will accelerate a workflow from 8 to 80 as quickly and effortlessly as a car would on an open road. From 80 to 99 is a different sport. That stretch is purchased with team shape and architecture, and no study of tools alone will ever show it... as tools alone will not get you there.

Run your own before-and-after

Use three numbers, spend one afternoon, and there is nothing to buy.

1. The completion ratio. Planned versus completed story points, the last four sprints compared to the four before adoption. Did completed go up while the ratio went down? That is the pattern from the paper: real gains, spent in advance by the roadmap.

2. The activity trap. Take a look at your main post-adoption AI chart. If it shows usage, commits, or lines, try to name one decision that it could correctly inform. In the paper, it would have scored a 59 percent gain as a zero.

3. The queue check. From last week’s prompt: median time-to-first-review and time-in-review, before and after the adoption. If both increased while throughput also increased, your constraint has moved downstream and is compounding quietly.

If your shapes match the paper’s, your engineers are fine. Something was waiting in your org chart, and it was there before the tools arrived.

AI makes the work denser, not bigger. Measure volume and you will see nothing happen... right up until the promises stop being kept.

Coming up

Next week I climb down into the architecture body: one modernization pattern in full, the GenAI-assisted legacy rebuild. What it costs, what it returns, where it goes wrong. The pattern, not the pitch.

More soon.:-]

Sources

Because none of this comes only from my head. Every link checked.

• The spine: Rafael Tomaz, Paloma Guenes, Allysson Allex Araújo, Maria Teresa Baldassarre, and Marcos Kalinowski, “Impacts of Generative AI on Agile Teams’ Productivity: A Multi-Case Longitudinal Study“ (FORGE ‘26, IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering, 2026). Peer-reviewed. A small sample watched for a long time, and honest about its limits. The dataset is open on Zenodo.

• The five-dimension lens the study uses: Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler, “The SPACE of Developer Productivity“ (ACM Queue, 2021). Practitioner venue, not peer-reviewed in the journal sense.

• The lab-speed antecedent the paper builds on: Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer, “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot“ (arXiv 2302.06590, 2023). Controlled experiment, preprint, not peer-reviewed: 55.8 percent faster on a fenced task.

• The perception gap, again: Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity“ (METR, arXiv 2507.09089, July 2025). Randomized controlled trial, preprint, not peer-reviewed.

• Herbie and the hour at the bottleneck: Eliyahu M. Goldratt and Jeff Cox, “The Goal: A Process of Ongoing Improvement“ (North River Press, 1984). Still not peer-reviewed. Still right.

I make this Substack thanks to readers like you!

Subscribe now

It is a dashboard: AI usage by engineer, tokens per week, sorted descending. The manager scrolls to a name near the bottom of the list, and the question is almost kind. “Why is your number so low?”

Two desks over sits an engineer whose number is magnificent. He wears that dashboard the way other people wear a marathon medal. And in the same team’s repo, the oldest pull request has been waiting for a reviewer longer than either of them has been tracking tokens.

I have watched versions of this meeting more than once. The company changes, the tool changes… the dashboard survives.

Everyone in that room believes they are doing AI transformation. What they are actually doing is maximizing the input to the one step of delivery that was never the problem.

The mandate says: use the model more.
The delivery data says: your bottleneck doesn’t burn tokens.

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 4 climbed inside the AI capability and asked for one number: the acceptance rate.

This week, as promised: the team body. What the org chart has to give up for the model to stay in the loop.

And for the first time, there is a kit. Near the end you will find a five-question team-shape test and a prompt you can paste over your own delivery data, today, with the tools your mandate already bought.

If you run it, subscribe and tell me what your Herbie turned out to be. Herbie is explained below, and he matters more than your token count.

Subscribe now

The mandate arrived with a leaderboard

In April 2025, Tobi Lütke told Shopify that “reflexive AI usage is now a baseline expectation” and wired AI competency into performance reviews and hiring. A few months later, Brian Armstrong told the Cheeky Pint podcast that Coinbase bought Cursor and Copilot, gave engineers a week to onboard, and fired the ones who did not. Coinbase now says roughly a third of its code is written by AI, aiming to have it at 50%…

Let me be fair to the memos before I take them apart. Refusing to seriously try these tools in 2026 is a career decision, and a bad one. If the mandate stopped at “try it, properly, this quarter”, I would co-sign it.

But the mandate never stops there, because an executive who mandates something needs to measure it… and usage is the number that is easy to get. Seats. Active users. Tokens per engineer per week. Somewhere down the org chart, that number becomes a sorted list… that list becomes a 1:1 conversation and even a way to evaluate “performance”.

All this knowing that measured people have always done: they optimize the metric. Pipe the model into everything, let it summarize what nobody reads, regenerate until the meter looks respectable. Call it tokenmaxxing. Burning tokens is the one KPI an engineer can max without shipping anything and some wear it as a badge of honor. A leaderboard where the prize for winning is a bigger invoice.

Which is exactly where the story has now arrived. The same companies that mandated usage are staring at their AWS Bedrock, Anthropic and OpenAI bills and asking where the value went.

They mandated an input… they got the input…
And now they act surprised because input is not the same as value created :-D

Issue 4 called usage the vanity metric: the treadmill is plugged in, nobody is running. This issue is about something worse. Even if everyone runs, maximizing this particular input cannot move delivery, even in principle. To see why, you need a book that is older than everyone building those dashboards’ careers.

Herbie doesn’t type

In 1984, Eliyahu Goldratt published “The Goal”, a novel about a factory manager (which is a sentence I promise gets better). For the academices out there, it is not peer-reviewed… it is barely literature. At the same time, it is also the clearest thing ever written about why your AI mandate is not working.

The famous scene is a boy scout hike. The troop keeps stretching apart because one kid, Herbie, is slow. The fast kids sprint ahead, then wait. The line’s speed to the destination is Herbie’s speed, no matter how impressive everyone else’s pace looks. The fix is not to yell at the fast kids to walk faster. It is to put Herbie at the front, take the extra weight out of his backpack, and subordinate everyone’s pace to his.

Out of that hike comes the Theory of Constraints and its two brutal accounting rules:

An AI usage mandate is a jetpack for every scout except Herbie.

Because in software delivery, typing code was almost never Herbie. Ask where a change actually spends its calendar life. Waiting for review. Waiting for the meeting where two teams agree on the interface. Sitting in the handover between the team that wrote it and the team that operates it. Or worst of all, being rebuilt, because what shipped was not what the customer needed and finding that out took a full cycle. The constraint lives in the seams between people. It always did.

Now make code creation dramatically faster, company-wide, by decree. What happens? The queue in front of Herbie gets longer. That is the entire story of enterprise AI adoption in one sentence, and Goldratt priced it forty years before the first token was billed.

The data agrees, awkwardly

Two pieces of evidence, one small and sharp, one large and blunt.

The sharp one: Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein at METR ran a randomized controlled trial (July 2025) with 16 experienced open-source maintainers doing 246 real tasks in repos they knew deeply. Before starting, the developers forecast AI would make them 24% faster. Afterwards, having lived it, they estimated it had made them 20% faster. The stopwatch said they were 19% slower.

Sixteen developers, early-2025 tools, unusually mature codebases: carry the number with care. But carry the shape of it everywhere, because the gap between felt speed and measured speed is the fuel every usage mandate runs on. If the person doing the work misreads their own speedup by nearly 40 points, the executive reading a token dashboard three levels up has no chance.

The blunt one: the 2025 DORA report (Google’s research program, vendor-funded, not peer-reviewed, and honest about both) finds that AI adoption now correlates with higher delivery throughput and with higher instability at the same time. More change ships, more change fails and the pile-up happens exactly where Goldratt said it would: review, testing, QA. Their respondents say reviewing AI output is harder than writing code, that time saved writing is re-spent auditing, and 30% of developers report little or no trust in AI-generated code. DORA’s own framing is that AI is an amplifier: it does not remove your bottleneck, it delivers more traffic to it.

I'd be reluctant to follow these studies as they are 2025 dated and data-bounded to that era… while we all know how much has changed since Opus 4.6 and the whole Agentic AI boom in the last 8 months. Nevertheless, I see and hear these arguments and feelings in every customer…

Which brings me to the promise I made last week, about the acceptance rate that climbs for a quarter and then quietly slides back.

Last week's edition argued acceptance rate is the honest number, and it is. Honest about the model. It is not honest about the system, because when the constraint sits downstream of the model, the number decays in one of two ugly ways. Either reviewers drown and start waving things through, so acceptance climbs while your change-failure rate climbs with it. Or reviewers hold the line and become the queue, so acceptance stalls, your best senior turns into a full-time auditor, and adoption slides back to whatever the team could actually digest. Nothing about the model changed in either case. The team shape did the deciding. That is what a team-shape problem wearing a model costume looks like from inside.

What the org chart has to give up

Goldratt’s method, compressed: find the constraint, get the most out of it, and subordinate everything else to it. Subordinate is the step where org charts flinch, because subordination costs status. Here is what it costs, concretely.

Give up the usage league table. Replace tokens-per-engineer with flow at the constraint: time to first review, time in review, age of the oldest open PR. You cannot manage Herbie with a dashboard that only shows the fast kids. Measure the queue, not the faucet.

Give up output-per-person as the status currency. If review is your constraint, then review capacity is first-class engineering work, not a tax on the seniors’ “real” job. The engineer who reviews fast and well is not underperforming on code output. They are carrying Herbie’s backpack, and the career ladder has to say so out loud, or nobody sane will keep doing it.

Give up the single gate. One architect who must bless everything, one platform team every change routes through, one codeowner file that funnels a whole repo to one person: these are constraints you built on purpose and forgot about. Redraw ownership so one team can take a change, including the model’s share of it, from start to shipped without leaving the room. This is the argument of Skelton and Pais’s “Team Topologies”: teams are the unit of delivery and their cognitive load is a budget you design for, not a personal failing.

Give up the ceremony. Alignment that lives in standing meetings and handover documents is the constraint wearing a calendar costume. Every handover is a queue. Every weekly sync that exists because two teams share a seam is Conway’s law sending you an invoice. Remember from last week that the model reads your architecture like an honest new hire? Your meeting load is the same reading, of the team body this time.

Notice what is not on this list: a bigger model, a better prompt, one more token. The 8% to 80% story I have carried since the first Substack post was never a model story alone. That step only held at 80 because the team around it changed shape at the same time, ownership included. The model got it to possible. The org chart got it to Tuesday.

Run the team-shape test

Five questions, yes or no. Count your yeses.

1. Is your primary AI metric a usage number (seats, active users, tokens), viewable per person?

2. Take the last meaningful change your team shipped: did it spend more calendar time waiting (for review, for approval, for a meeting) than being built?

3. Can you name, without thinking, the one person or team most changes queue behind?

4. Has an AI metric (usage or acceptance) climbed and then slid back within two quarters, with no model change to blame?

5. If every engineer’s AI usage doubled tomorrow, would the first thing to break be a human queue: review, QA, or a release approval?

0 to 1 yes: your constraint may genuinely be code creation. Mandate away, you strange and fortunate shop.
2 to 3 yeses: mixed picture. Start measuring the queues this week, before the next dashboard review measures the wrong thing again.
4 to 5 yeses: you have a team-shape problem wearing a model costume. Your org chart owes Herbie an apology, and the next section is for you.

The prompt

Paste this into whichever assistant your mandate bought you, and feed it real data. It takes about twenty minutes to collect and one meeting to act on.

You are helping me find my software delivery constraint. Be blunt. Do not flatter me.

I will give you three inputs:

1. My team's last 20 merged PRs/MRs, each with four timestamps:
   opened, first review comment, approved, merged.
   [paste]

2. One typical week of the team calendar: each recurring meeting,
   its length, and how many people attend.
   [paste]

3. Our AI usage snapshot, if one exists: seats, active users,
   tokens or suggestions per week. If none, I will say "none".
   [paste]

Compute, per PR: time to first review, time in review, and time from
approval to merge, each as a share of total open-to-merge time.
Use medians, not averages.

Then answer three questions:
- Where does a change spend most of its calendar life: being written,
  waiting for a human, or waiting for a meeting?
- Who or what is my Herbie: the single queue the median change waits
  in longest?
- If my team's AI usage doubled tomorrow and nothing else changed,
  which queue grows first?

End with: the one team-shape change most likely to cut the median
open-to-merge time, and the one metric I should stop reporting.

The model will happily run this analysis. It is, after all, already in the room.

A mandate can make everyone use the model. Only the org chart can make it matter. Feed the constraint, not the meter.

Coming up

Next week I open the field notebook: what it actually looks like when a shop moves its constraint, all three bodies at once, and the before-and-after numbers that survived contact with reality.

More soon. :-]

Sources

Because none of this comes only from my head. Every link checked.

• The constraint, Herbie, and the mirage: Eliyahu M. Goldratt and Jeff Cox, “The Goal: A Process of Ongoing Improvement“ (North River Press, 1984). A business novel, not peer-reviewed, and still the sharpest operations book on this list.

• The Shopify mandate: Tobi Lütke, “Reflexive AI usage is now a baseline expectation at Shopify“ (internal memo, self-published on X, April 2025). Primary source.

• The Coinbase mandate: Brian Armstrong on Stripe’s Cheeky Pint podcast, as reported by CIO, “Fire any developer who doesn’t use AI” (September 2025). Press coverage of a podcast, not peer-reviewed.

• The perception gap: Joel Becker, Nate Rush, Elizabeth Barnes, and David Rein, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity“ (METR, arXiv 2507.09089, July 2025). Randomized controlled trial, preprint, not yet peer-reviewed.

• Throughput up, instability up: the 2025 DORA report and its “Balancing AI tensions” insight (Google Cloud’s DORA research program). Large-scale survey, vendor research, not peer-reviewed.

• Teams as the unit of delivery: Matthew Skelton and Manuel Pais, “Team Topologies“ (IT Revolution Press, 2019). Practitioner framework, not peer-reviewed.

I make this Substack thanks to readers like you!

Subscribe now

The slide says the AI capability is delivered. Seats are bought, the usage graph climbs, and someone has drawn a green checkmark next to the third body. On paper, the model is in your pipeline.

Then you look at where the work actually happens. The model drafts, and a person quietly rewrites most of it before anything ships. It suggests, and the suggestion gets read, sighed at, and closed. The tool is open on every screen and the job is being done the same way it was 6 months ago.

That model is present. Present is not the same as in the loop.

Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put the AI capability on the table as one of three bodies that move together.

This week I climb inside that body and ask the only question that matters once the model is bought: is it in the loop, or just in the room?

A model is in the loop only where you would ship its work without redoing it. Everywhere else it is in the room: adopted, visible, paid for and not actually doing the job.

No kit this week. Just the argument and the standing ask: if you have put a model into a real pipeline, subscribe and tell me where I am wrong.

Subscribe now

The model as a build lever

The first thing a model in the loop does is change what is worth building. Not how fast you type. The size of the thing one person can now finish.

Every backlog has a graveyard: the work that was permanently parked because the payback never cleared the bar. The manual review step no one would fund a rewrite for. The migration that needed six months of a team you were never going to get. A model in the loop moves the economics, not just the ergonomics, and some of that graveyard work suddenly clears the bar. The item that was “too expensive to bother with” becomes a Tuesday.

That is the lever. It is quieter than the demos suggest and more consequential. It does not make your existing plan faster. It changes which plan is worth having, which means it reaches straight back into the architecture body and the team body, the same coupling Issue 3 was about. Treat the model as a faster autocomplete and you will use a build lever to shave minutes off typing, which is roughly using a forklift to carry your lunch.

The model as an architectural mirror

The second thing a model in the loop does is show you your architecture, whether you asked or not.

A model can only act where your system lets it act. Point it at a clean seam with a real contract, and it moves: it reads the boundary, does the work, hands back something you can ship. Point it at a tangle where the batch job, the shared database, and the nightly cron all reach into each other, and it stalls, hedges, or invents. It cannot find the edge of the thing you are asking it to change, because there is no edge.

So where the model reliably fails, it is usually not failing. It is reflecting. A model that keeps getting the same file wrong is often telling you that file does three jobs. That is Conway’s law with a faster feedback loop: Melvin Conway said in 1968 that a system’s design mirrors the communication structure that built it, and now the mirror talks back in an afternoon instead of a reorg. The model reads your architecture the way an honest new hire reads it, out loud, on day one, without the tact. Most teams experience that as the AI being bad at their codebase. It is worth asking, at least once, whether the AI is being accurate about their codebase.

The wall chart of the future cannot tell you

There is a genuinely useful map of this territory, and I want to credit it before I argue with it. Amalfitano and ten co-authors, in a 2026 ACM TOSEM roadmap, sort GenAI in software engineering along two honest questions: what is being augmented (your process or your product), and how autonomous it is (does it wait for you, or act on its own).

That gives four clean forms, the Copilot, GenAIware, the Teammate and the Robot. As a shared vocabulary that beats the usual “AI for SE” hand-waving, and I will happily use it.

Then the same paper closes with ten predictions for 2030, and the precision is where it gets funny.

• 70% of boilerplate autonomously generated

• “Prompt Architect” a certified job title

• 40% of job postings requiring it

• 20% of teams in regulated industries carrying a dedicated role.

These are not ranges or scenarios. They are two-significant-figure forecasts for a field that cannot reliably tell you what next quarter’s model will cost. And in the acknowledgments, the authors note the predictions were brainstormed with Claude Sonnet 4.5, off a literature review filtered by two other models. A roadmap for GenAI in software, whose boldest numbers were generated by GenAI.

The taxonomy is real work, so I say this without malice: the map of the future is cheap to generate and getting cheaper, and it will never tell you whether the model is in your loop this week.

The gap between present and in-the-loop shows up one layer down too. Moonshot just released Kimi K3 with open weights, and plenty of people read “open weights” as “we can run this ourselves now.”

The weights are about 1.5 terabytes and need multiple top-end GPU nodes to serve. Downloadable, yes. Runnable on your very expensive macbook or even a server under your table, no.

And a model can top the public leaderboard and still not be the one a team ships with, because fit to the actual work, the cost of a wrong answer, and where it sits in the pipeline decide more than the benchmark does.

Route the cheap work to the cheap model, keep the expensive judgment where it earns its keep, and stop confusing a model you can name with a model that is doing your job.

Present is not in the loop, at the model layer and the org layer both.

The one number

So how do you tell present from in-the-loop without a slide? One number: the acceptance rate of the model’s work. The share of what it produces that you ship without a human redoing it.

Not seats. Not tokens burned. Not “queries this month,” which measures that people opened the tool, not that the tool did the job. Usage is the vanity metric of AI adoption - it goes back to when we started doing BI dashboards and competed which teams had more.

Acceptance is the outcome. A usage dashboard tells you the treadmill is plugged in. Acceptance rate tells you whether anyone is running.

This is not just my framing. Albert Ziegler and colleagues, in “Productivity Assessment of Neural Code Completion” (MAPS 2022, expanded in Communications of the ACM in 2024), found that the acceptance rate of a model’s suggestions predicted developers’ own sense of productivity better than the other metrics they measured. The thing you keep is the thing that counts.

Which is why the one number I carry from the earlier issues is an acceptance number. That manual review step that went from roughly 8% machine-handled to 80% was never “the model was consulted 80% of the time.” It was 80% of cases closed by the model’s output and shipped, no human redoing the work. The 72 points between those two states is the model moving from in-the-room to in-the-loop on that one step. Had the number instead been “80% of reviewers now have the AI tool open,” we would have been measuring furniture and calling it transformation.

Run the in-the-loop test on your own pipeline

You can tell present from in-the-loop in about 3 questions.

1. Can you say your acceptance rate today, for one real step, as a number? If the only figures you have are seats and logins, you are measuring the room, not the loop.

2. Where the model stalls, is that a model limit or an architecture seam? Point it at your worst file and watch. If it keeps missing, the file is probably doing three jobs, and the model just told you which three.

3. What would you ship from the model without redoing it? If the honest answer is “nothing yet,” the model is present, and you have a build-lever and a mirror problem to fix before you have an AI capability.

Run it on your own pipeline before the next planning cycle, not on a vendor’s demo after.

A model is in the loop only where you ship its work without redoing it. Everything else is furniture with a login.

Coming up

Next week, the body nobody puts a number on: the team. What the org chart has to give up for the model to stay in the loop, and why an acceptance rate that climbs for a quarter and then quietly slides back is almost always a team-shape problem wearing a model costume.

More soon. :-]

Sources

Because none of this comes only from my head. Every link checked.

• The taxonomy and the 2030 predictions: Domenico Amalfitano, Andreas Metzger, Marco Autili, Tommaso Fulcini, Tobias Hey, Jan Keim, Patrizio Pelliccione, Vincenzo Scotti, Anne Koziolek, Raffaela Mirandola, and Andreas Vogelsang, “A Research Roadmap for Augmenting Software Engineering Processes and Software Products with Generative AI“ (ACM Transactions on Software Engineering and Methodology, 2026). Peer-reviewed. The four forms are in the classification section; the ten predictions and the note that they were brainstormed with Claude Sonnet 4.5 are in the concluding section and acknowledgments.

• Acceptance rate as the signal: Albert Ziegler, Eirini Kalliamvakou, and colleagues, “Productivity Assessment of Neural Code Completion“ (Proceedings of MAPS 2022, the 6th ACM SIGPLAN International Symposium on Machine Programming), expanded as “Measuring GitHub Copilot’s Impact on Productivity” (Communications of the ACM 67, no. 3, 2024, pp. 54-63). Peer-reviewed.

• The org mirror: Melvin E. Conway, “How Do Committees Invent?“ (Datamation, 1968). The origin of Conway’s Law.

• Kimi K3 open weights and hardware footprint: Moonshot AI’s Kimi K3 open-weights release (July 2026), reported by The Week and others; the roughly 1.5 TB weight size and multi-node serving requirement come from the release and deployment coverage. Vendor release and press, not peer-reviewed.

I make this Substack thanks to readers like you!

Subscribe now

It is wrong before the first bar starts. Not because the order is wrong, but because there is no right order.

The three things on that wall are not three projects waiting their turn. They are one big sociotechnical system, and each part pulls on the other two the moment it moves.

Issue 1 named the failure: sequencing is the trap.

Issue 2 named the destination: AI-native, not AI-ready, and you buy AI-ready by default.

This is the machinery underneath both. The three bodies themselves, what each one is, and what “moving together” looks like on a real roadmap instead of a slide.

Subscribe now

No kit this week. Just the machinery, and the standing ask: if you have run one of these, subscribe and tell me where I am wrong.

Why three bodies, not three projects

Borrow the name honestly. In celestial mechanics, two bodies under gravity have a clean solution: you can write down where they will be forever. Add a third and it collapses. Henri Poincaré showed in the 1890s that the three-body problem has no general closed-form solution. Each mass pulls on the other two continuously, so you cannot compute one path while holding the others still. This is settled mathematics, not a metaphor I am stretching, and it is why we fly spacecraft through these systems with continuous course corrections instead of one equation.

That is your transformation. The three bodies are the architecture (the system you are rebuilding), the teams (the org that owns it: topology, cognitive load, decision rights), and the AI capability (what the model now makes buildable, and how it reshapes the other two). Move one and the other two move, whether you planned for it or not. Rebuild the platform and the team that owned the old one is now shaped for a system that no longer exists. Put a model in the loop and what is even worth building changes, and the team shape with it.

Sequencing assumes you can freeze two bodies while you work the third, then unfreeze them. You cannot. Nothing holds still while you take your turn.

Body one: the architecture

These are your systems being constantly stressed and rebuilt, and it is the one everyone books first because it feels like the foundation. It is also the one that yanks the other two hardest. A real modernization does not just swap the runtime. It changes where work happens, which means the team that grew up around the old system is now organized for a shape that no longer exists, and the AI you were going to add later suddenly has a very different surface to attach to. You cannot finish this body and hand a stable artifact to the next program. The moment it moves, it has already reached into the other two.

Body two: the teams

Melvin Conway said it in 1968 and it has never stopped being true: a system’s design mirrors the communication structure of the org that built it. Move the architecture and leave the org boxes where they were, and the old structure quietly pulls the new system back into its shape. This is the reorg that does not take, the one that gets celebrated at launch and reverts by the next quarter.

Team Topologies (Matthew Skelton and Manuel Pais, 2019) gives this a handle: organize teams around the flow of the work and manage cognitive load deliberately, or the load lands on people anyway, uninvited. The team body is not an HR exercise you run after the platform ships (or worse, an excuse for a reorg). It is the difference between a system that holds and one that erodes back into the org chart that predates it.

Body three: the AI capability

The third body is not a library you consult. It is a capability in the loop where the work actually happens, in how code gets written and reviewed, how a platform validates itself, what a team can take on. And it does not sit politely inside the box you drew for it in FY3. It reshapes the other two. It changes what is worth building at all (that is the architecture body moving again) and how many hands a thing takes (that is the team body moving again). Bolt it on at the end and you get AI-ready: a structure that can hold AI, sitting next to a system that never changed to use it. Put it in the loop from the start and it pulls its weight, and pulls on everything else.

Moving together is a verb, not a Gantt chart

Here is the part the wall chart cannot show. Steering three coupled bodies does not look like three bars in a row. It looks like one program that replans continuously, because every time one body moves you re-solve the other two. A platform decision changes the team plan this week, not next year. An AI capability lands and the backlog it was supposed to accelerate turns out to be the wrong backlog, so you change what you are building.

This is also why the vendor split is so expensive. Run modernization with a systems integrator, the org change with a transformation firm, and the AI with a separate consultancy, and you have handed the three coupled bodies to three parties who each optimize their own and none of whom own the coupling. They do not compound into capability. They compound into dysfunction, on three invoices. The alternative is not a bigger program. It is one operator holding all three, with no hand-offs across the seams where the pull actually happens.

The smallest complete example, the whole thing in motion

The cleanest version I have run looked like this. One mandate that, on a normal roadmap, would have been three projects in a line. We rebuilt a batch platform into one that ran in real time. We restructured the team around the flow of the work instead of the old function boxes. And we put a language model into a manual review step, the kind where people used to grind through a queue by hand, and took it from roughly 8% machine-handled to 80%.

Told as three programs, that is a modernization, then a reorg, then an AI pilot, each waiting on the one before it, and the review step stuck at 8% the whole time while everyone waits their turn. Told as one program, it is three bodies moving together: the platform made real-time work possible, the team shape made the platform hold, and the model in the loop was the thing that made the whole rebuild pay. Not the AI beside the process. The AI as the process, on a platform and a team reshaped to own their flow of work.

Run the coupling test on your own program

You can tell steering from sequencing in about three questions:

1. How many budgets and owners do the modernization, the org change, and the AI sit under, and on whose timeline? Three separate ones, each with its own year: you are sequencing.

2. What happens to the team plan when the architecture slips a quarter? If the answer is “nothing, that is a different workstream,” the bodies are not coupled on paper, which means they will collide in practice.

3. When does the AI capability get to change what you build, not just how fast? If the answer is “after the platform is done,” the third body is bolted on, not in the loop.

Run it on your own program before the next planning cycle, not on a vendor after.

The whole thing, in one line

You do not sequence a system whose parts pull on each other. You steer it. Together, or it does not really move at all.

Coming up

Next week I go hands-on inside one body, the AI capability, and what “in the loop” actually means in the delivery pipeline: the model as a build lever, the model as an architectural mirror, and the number that tells you it is working instead of just present.

More soon. :-]

Sources

Because none of this comes only from my head. Every link checked.

• The physics: the three-body problem’s lack of a general closed-form solution is due to Henri Poincaré (1890s); a standard scholarly treatment is June Barrow-Green, Poincaré and the Three Body Problem (American Mathematical Society, 1997). Scholarly / peer-reviewed.

• The org mirror: Melvin E. Conway, “How Do Committees Invent?“ (Datamation, 1968). The origin of Conway’s Law.

• The team body: Matthew Skelton and Manuel Pais, Team Topologies (IT Revolution, 2019). Practitioner book, not peer-reviewed.

• The operating-model failure mode (callback to Issue 2): Mik Kersten, Output to Outcome (IT Revolution, 2026). Practitioner book.

I make this Substack thanks to readers like you!

Subscribe now

The internal assistant answers a question about the travel policy. The “AI knowledge base” comes up on the big screen and people are genuinely proud of it. A slide says the pilots will save 4k hours/year and that number came out of the vendor’s ROI calculator, which has never met this company’s actual work and is therefore free to be optimistic.

And the delivery pipeline, the thing that turns an idea into software running in production, has not changed a single stage since before any of this was bought.

That company is AI-ready. It believes it is AI-native. The real story of the program is not whether the AI works, it is that nothing about how the company operates changed around it.

Last week I closed with a tell: if your proudest AI artifact is a reference library and your pipeline has not changed a stage, you already know which one you bought. This is that essay, the one Issue 1 existed to set up:

Subscribe now

No kit this week either. Just the argument, and the standing ask: if you have run one of these programs, subscribe and tell me where I am wrong. The interesting version of this idea is the one that survives contact with your actual estate.

AI-ready is what the money buys by default

I keep walking into the same project. A company decides this is the year of AI. It buys the licenses, stands up an internal assistant, writes the knowledge base, runs a few pilots, and puts a number on a slide. On paper it is an AI company now. The paper is doing a lot of work.

Underneath, the operating model is exactly what it was the year before. The estate is the same. The way work moves from idea to production is the same. The AI is sitting next to the system, not inside it.

That is AI-ready: a structure that can accommodate AI. It is not nothing. The licenses are real, the assistant answers questions, the knowledge base is genuinely tidy. But it is not what the buyer thought they were paying for, and it is not where the value is.

AI-native is a difference in kind, not degree

AI-native is an operating model built around AI as a live capability in the workflow. The model is in the loop where the work actually happens: in how code gets written and reviewed, how decisions get made, how the platform validates itself. Not a library you consult. A capability you run.

A word on the words, because a book already uses some of them. I am not trying to claim “AI-native” as mine. Melissa Reeve’s Hyperadaptive: Rewiring the Enterprise to Become AI-Native (IT Revolution, May 2026) is built on the term, and plenty of vendors have discovered it fits nicely on a homepage. The term is cheap. The how is scarce. Own the how, not the label.

Mik Kersten just gave this failure its proper name

Mik Kersten’s new book, Output to Outcome: An Operating Model for the Age of AI (IT Revolution), is out today, and it lands squarely on the mechanism: organizations get very good at producing output and never move the outcome.

AI-ready is that failure with a bigger invoice. You ship more output than ever, an assistant, a knowledge base, a dozen pilots, and the outcome the business actually feels sits exactly where it sat the year before. The victory-lap meeting is a celebration of output. The outcome was not invited.

When I talked to Mik about this essay, he pointed me at Chapter 7, “Matrix to Modularity”, and he was right to. It is this argument run from the organizational end. “AI and agentic solutions can create tremendous output, especially when working within modular structures,” he writes. Most incumbents are not modular. They are a matrix of dependencies, dotted reporting lines, and meeting cadences layered over meeting cadences, and the AI gets bolted onto exactly that.

His answer is a concept called sociotechnical congruence: design the organizational structure and the technical structure to fit each other, on purpose, because “even with the best architectural intentions, a tangled organizational structure will create a tangled technical architecture.” Readers of Issue 1 will recognize this. The sociotechnical wiring I complained was missing from the operating-model textbooks just showed up in one. It went on sale this morning.

Kersten has been pulling on this thread since Project to Product, and this is the sharpest frame I have found for why that meeting feels so hollow. If you are going to read one book on the operating-model side of AI this quarter, make it this one.

Sequencing is how you buy AI-ready twice

The reason most transformations land on AI-ready is that they sequence. Modernize the architecture this year. Reorganize the teams next year. Add AI on top after that. Three programs, three budgets, three sets of consultants, each a success on its own terms. And AI-native falls out the bottom, because by the time the AI arrives, the architecture it lands on is a year stale and the teams are still shaped for the old flow.

You do not get to AI-native by sequencing. The architecture, the teams, and the AI capability pull on each other. Change the architecture and the teams that own it have to change. Put AI in the loop and what is even worth building changes again, and the team shape with it. They move together or they do not really move at all.

For a European buyer the framing that matters is cost and risk, not growth. The cost of AI-ready that looks AI-native is the second migration. You spend the budget, you get the assistant and the knowledge base, and eighteen months later the same gap is still open, because the operating model never moved. Then you pay again. The expensive path is not doing the three together. The expensive path is doing them one at a time and discovering you have to redo the first by the time you reach the third.

The smallest complete example I have

The cleanest version of the alternative I have run looked like this: one mandate that, on a normal roadmap, would have been three projects. We rebuilt the data platform, restructured the team around the flow of the work rather than the old function boxes, and put an LLM into the validation step where analysts used to grind through a queue by hand.

On a good engagement, that one-pass version is the difference between a grinding manual step running at roughly 8% machine-handled and the same step at 80%. Sequenced, it stays at 8% three times in a row, once per budget.

Not three projects in a line. One. The AI was not beside the process. It was the process, on a platform and a team that had been reshaped to hold it. That is AI-native in miniature.

The five-minute tell, ready to run

You can spot AI-ready pretending to be AI-native in about five minutes. Three questions, and you can ask them in any order:

1. Ask to see the proudest AI artifact. If it is a knowledge base or a reference kit, mark it.

2. Ask what changed in the delivery pipeline since the AI program started. Same stages, same hand-offs, same lead time: mark it.

3. Ask where the ROI number came from. A generic calculator instead of a measured before-and-after on real work: mark it.

Three for three and you bought the structure, not the capability.

Run it on your own shop before the next AI budget goes out, not on a vendor after. It is less fun that way, and considerably more useful.

The whole distinction, in one line

AI-ready is a structure that can hold AI. AI-native is an operating model built around it. The invoice looks the same. The year after does not.

The good news is that the fix is not more AI. It is moving the three bodies, the architecture, the teams, and the AI capability, at once.

Coming up

Next week I open up the machinery: the three bodies themselves. What each one actually is, what “moving together” looks like on a real roadmap instead of a slide, and one concrete example of the whole thing in motion.

More soon. :-]

Sources

Because none of this comes only from my head. Every link was checked.

• The frame: Mik Kersten, Output to Outcome: An Operating Model for the Age of AI (IT Revolution, 2026), released 14 July 2026. Quotes are from Chapter 7, “Matrix to Modularity”, in the Kindle edition. His earlier Project to Product (IT Revolution, 2018) started the thread.

• The term: Melissa M. Reeve, Hyperadaptive: Rewiring the Enterprise to Become AI-Native (IT Revolution, 2026).

Both are practitioner books, not peer-reviewed research. That is fine. So are these field notes.

I make this Substack thanks to readers like you!

Subscribe now

An analyst opens a validation queue every morning and works it by hand. Check the figure, check the source, flag the ones that smell wrong, escalate the rest. Skilled work, and most of it deadening.

I got to take that apart in different customers. We rebuilt the platforms underneath it, restructured the teams around the flow of the work instead of the old function boxes, and use AI when needed while prioritizing good software engineering practices. And not a single time we managed to fully automate the processes... we could speed up things, reduce toil... and that is what I want to keep on doing.

Here is the part that matters, and it is not the model.

It never was only about three projects, but always about 1 mandate. Rebuild the systems, reshape the teams that owns it, put AI in the loop when it makes sense and where the work actually happens, all at once… because any one of them alone moves the number by roughly nothing.

Everyone wants to talk about the GenAI model (“open weights is better, while the new claude/gemini/anthropic is this and that”) ... but that is the easy part. The hard part, the part that on a good engagement moves a grinding manual step from roughly 8% handled by a machine over 1 week of work to 80% automagically going in 2h, was that the platform, the team and the model were built to fit each other in a single pass. Do those in sequence, a year apart, three budgets and three sets of consultants, and you get three tidy successes and a number that never budges.

That gap is the thing I keep coming back to. It is the whole reason for this.

Who is writing this

I am Thiago. Twenty years ago I was a statistician in Brazil, then a data scientist, a DBA, a software engineer, data engineer, a hands-on architect, then a CTO… these days a recovering CTO who still writes the code and knows that systems design while keeping myself at the trenches is what makes me happy. The recovering part is that I have run orgs internally, sat as interim head of _fill the gap_, been on the hook for the quarterly number and still never managed to fully close the loop only with tech.

Most of my recent years went into modernization and into getting generative AI into how software actually gets built, not into slides about how software might one day get built.

This is issue one. Every week I am going to work one problem out loud here: architecture modernization, team design, and AI capability as a single program, written by someone still in the code and not narrating it from a stage.

No downloadable kit this week. No five-box framework in a tidy PDF. Just the argument, and an ask. If you have lived this, subscribe and tell me where I am wrong. That is not a growth tactic, it is genuinely half the point, because the interesting version of this idea is the one that survives contact with your actual estate.

Subscribe now

Everybody already has an operating model. Almost nobody wrote it down.

Every company I walk into already has an operating model, whether or not anyone calls it that. It is just how IT creates and delivers value: who owns what, how work moves from an idea to something running in production, where the decisions actually get made versus where the org chart says they do.

Most companies never wrote it down. The ones that did wrote a box diagram, and the box diagram has not been opened since the offsite where it was drawn.

None of that is wrong, exactly. It is just not the part that decides whether you ship. The part that decides is whether the architecture, the teams, and the AI capability were built to fit each other, or merely filed in the same org chart and left to work it out.

The textbooks are winging it too

I went and read the research on this properly, because I wanted to know whether the academics had a cleaner answer than I did.

They do not, and I can point you to the receipt.

In 2025, Suleman, Pekkola, Ralyté and Ahola published a systematic review in Information Systems Management that went through 77 academic and consulting sources hunting for an agreed definition of the IT operating model. It came back without one. The number of components the thing is supposed to have swings from four to thirteen, depending mostly on which consultancy is holding the pen. It is open access, so you do not have to take my word for it: the link is in the Sources at the bottom, go argue with it yourself.

The people whose actual job is to define this cannot agree on what it is or how many pieces it has. I find that oddly reassuring. If you have ever sat in a transformation kickoff quietly unsure what “operating model” meant in that room, relax: neither does the literature.

They drew the box. They skipped the wiring.

Here is the more useful finding from that same review, and it is the one that explains your last reorg.

The research treats the operating model as a structure. Decision-rights grids. Boxes for standardization versus integration. Who reports to whom. All of it about shape.

The one principle that would make the thing actually work is mostly missing: designing the technical side and the human side to fit each other, on purpose, at the same time, instead of optimizing one and inheriting the other. That idea is not mine either. It is called joint optimization, and it has been around since Trist and Bamforth’s coal-mining studies in 1951. Somehow it never explicitly made it from the sociotechnical-systems literature into the operating-model textbooks. Tune the tech, take the team as given. Or reshuffle the team, leave the systems where they were. Either way you optimized one subsystem and let the other one ride along as a hostage.

That is why the last reorg did not take. You moved the boxes and the systems stayed exactly where they were. And it is why the AI pilot is sitting next to the work instead of inside it, technically running, in the sense that a treadmill in a spare room is technically a gym.

There is one idea the academics keep underusing that every engineer already lives: your system ends up shaped like your org chart. Conway named it in 1968, in a paper literally titled “How Do Committees Invent?”. Skelton and Pais turned it into a practice in Team Topologies. Org structure and system architecture are the same problem, which is exactly why you cannot move one and expect the other to hold still.

On AI, the theory has not shown up yet

Now put AI into that picture, and it gets funnier.

The most cited frameworks for an AI-era operating model are not peer-reviewed. MIT CISR put out a research briefing on it in late 2025 (Thorogood and Woerner, “Enterprise IT Operating Models in the AI Era”). The fashionable “agentic operating model” comes from a 2026 California Management Review Insights column (Saini, “Governing the Agentic Enterprise”), which is editor-screened online commentary, not a peer-reviewed journal article. Both are thoughtful. Both are essentially field notes with good production values. Nobody has settled research on how AI actually reshapes the operating model. Practice is ahead of theory here, which is a generous way of saying we are all making it up, and the only real difference is who is keeping honest notes.

That greenfield is exactly what this newsletter walks into.

And it matters more than it sounds, because AI does not stay a tool for long. It becomes an actor in how your organization operates.

When Air Canada’s chatbot confidently invented a bereavement-fare policy that did not exist, the airline’s defense at tribunal was that the chatbot was “a separate legal entity responsible for its own actions.” They lost (Moffatt v. Air Canada, 2024 BCCRT 149, if you want to watch a tribunal politely decline to believe a company). They paid about 812 Canadian dollars, plus the legal time, which I promise cost more than that.

The lesson is not that the chatbot was dumb. The lesson is that when AI becomes part of how your operating model runs, the accountability does not get to spin off into a separate legal entity. It is still yours. You have quietly hired a coworker who can commit the company to things, cannot be fired, and will never once feel bad about it. Deciding where that coworker sits, and what it is allowed to promise, is an operating-model question. It is not a procurement one.

Four questions to run on your own operating model

You do not need a new framework for this. You need four honest answers about your own shop.

1. If you changed the architecture next quarter, would the teams that own it have to change shape? If the answer is no, those are not the teams that own it.

2. Where does your AI actually sit: in the loop where the work happens, or beside it, in a knowledge base people are proud of?

3. When the last reorg landed, did the systems change, or did the boxes move while the systems stayed put?

4. Name the number your AI moved. Not tokens processed, not seats licensed. The business number. If you cannot name it, you bought the structure, not the capability.

If those questions sting a little, good. That is the material.

The whole thesis, in one line

Architecture, teams, and AI are one program, or they are three expensive rehearsals. Three moving parts, one mandate, a number at the end.

That is what I am here to work out, in public, every week, with my hands still in the code.

Coming up

Next week I go straight at the piece that started all of this: the difference between being AI-ready and being AI-native, and why most transformations buy the first, call it the second, and pay AI-native money for the privilege.

Quick tell before then. If your proudest AI artifact is a reference library and your delivery pipeline has not changed a single stage, you already know which one you bought.

More soon. :-]

Sources

Because none of this comes only from my head. Every link below was checked.

• The definition problem: Suleman, Pekkola, Ralyté and Ahola, “IT Operating Models: A Systematic Review of Academic and Practitioner Literature,” Information Systems Management 43(1), 2025, 22-44. Open access, DOI 10.1080/10580530.2025.2513305.

• Joint optimization: Trist and Bamforth, “Some Social and Psychological Consequences of the Longwall Method of Coal-Getting,” Human Relations 4(1), 1951, 3-38, the founding sociotechnical-systems study.

• Conway’s Law: Melvin Conway, “How Do Committees Invent?,” Datamation, 1968. Turned into a practice in Matthew Skelton and Manuel Pais, Team Topologies (IT Revolution, 2019).

• AI-era frameworks (not peer-reviewed): Thorogood and Woerner, “Enterprise IT Operating Models in the AI Era,” MIT CISR Research Briefing XXV-12, December 2025; Sandeep Saini, “Governing the Agentic Enterprise: A New Operating Model for Autonomous AI at Scale,” California Management Review Insights, March 2026.

• Air Canada’s chatbot: Moffatt v. Air Canada, 2024 BCCRT 149 (British Columbia Civil Resolution Tribunal); the airline was ordered to pay 812.02 Canadian dollars in total (650.88 damages, plus interest and fees).

I make this Substack thanks to readers like you!

Subscribe now

Most companies now have an AI policy. Very few can show it working.

Teams ship code with agents in days, but review, testing and approvals still run at human speed… and nobody is sure who is allowed to decide what. Proof in the Pipeline is where I work that out in public: AI governance shown working where the code ships, not promised in a PDF.

So why is this so hard to fix? Because it’s a three-body problem. Architecture, teams and AI capability keep pulling on each other… move one and the other two move with it (hello, Conway). You steer all three together.

Where to start

• The idea: Steer, don’t sequence. Architecture, teams and AI capability pull on each other like three gravitating masses, so you steer them, you don’t sequence them.

• Where the bottleneck went: Your bottleneck doesn’t burn tokens and More shipped, same typing. AI sped up the step that was already fast; the numbers show where the queue moved.

• Governance in the pipeline: Who reviews the reviewers?, Prove it before it runs and Nobody owns the policy. Where the gate sits, and who owns which rule.

• Teams and agents: Conway’s law has a new employee, the start of the series on agent topologies as org charts.

Who’s writing: I’m Thiago de Faria, a data and AI architect who still writes the code, running TilinTheCloud from the Netherlands. More on the About page.

The weekly posts are free and stay free. If you want the code and research updates behind them, you can pledge a paid tier (nobody is charged until paid launches, and the founding AI Executive Circle has only 12 seats).

Subscribe, and tell me where I’m wrong :-]