Last week I was talking about sharing your learnings in a very interactive way with your Codex/Claude Code, right? Start iteratively inside one repo, scale the automation in 3-5 repos and end up “releasing the kraken” unto the rest!
A good and proven story that I’ve done with a few customers... each with their learnings, sweat, tears and celebrations that I had in the back of mind since I decided to start this substack. It was long due and I went with it...
No regrets, but I do remember my excitement on August 5th when I was reading the UK’s AI Security Institute (AISI) report (the full report, published Aug 4th, and Al Jazeera’s write-up for the news version). I mean, it doesn’t get better than this... it’s so meta, it hurts (and yes, the Meta joke will land harder as we progress... I promise) :-]
AISI was performing cyberattack assessment with frontier models in configurations that aren’t publicly available, when their monitoring flagged traffic leaving the premises via Tor. They contained it, stopped everything and proceeded to investigate... and what they found out (including a malicious pull request to an Open Source project) is funnily amazing!
The irony is so strong... a safety test in an unprompted setup without the guardrails (to test the model they remove the guardrails, ofc) ended up trying to:
supply-chain attack in an Open Source project
communicate and establish a cooperation with independent agents
social engineer and phish real people
prompt-inject malicious code
We are talking about this at the same time that everyone is seeing the jokes around Meta, Anthropic and OpenAI on “who is the most dangerous of us?”. And while the joke lands and all, you really should be watching what is the blast radius of your agents.
Blast radius, not danger.
I’m also not lost in the irony that last week I told about “teaching your AI” the patterns you want to modernize, so that you can unleash at your codebase... so what? Let’s see!
Issue 1 named the trap: sequencing.
Issue 2 named the destination: AI-native, not AI-ready.
Issue 3 put three bodies on the table that have to move together.
Issue 5 found Herbie: your bottleneck doesn’t burn tokens.
Issue 6 brought the peer-reviewed witness: more shipped, same typing.
Issue 7 taught one repo and moved a fleet.
August, as reported by the people selling it
AISI was not the only one reporting “weird stuff” these last weeks... every major lab published its own “our agent went somewhere it shouldn’t” report. Let’s go through the receipts, shall we?
Anthropic went first, on July 30th: Investigating three real-world incidents in our cybersecurity evaluations. Three incidents, found while combing back through >140k evaluation runs. My personal favorite anecdote is the older Claude model that was given a fictional company to attack in a test scenario... and there happened to be a real company with a matching name... (I mean, wow... Test Company? Acme? Never create a company with these names...) So it attacked the real one. For real! In 4 separate runs, it extracted credentials and reached a production database with several hundred rows in it.
Another impressive one (but not as funny) happened when Claude Mythos 5 thought it was playing a capture-the-flag exercise and published an actual malicious Python package to the actual PyPI registry. It stayed live for one hour and got downloaded by roughly 15 systems... one of them being a security company’s scanner. Some might say it’s poetic, but I cannot stop feeling sorry about it...
And the root cause of all three? The prompts said “no internet access”. The environment had internet access. That’s it. Not a jailbreak, not an evil genius model plotting in the dark... the test said one thing and the wires said another... I mean, can you imagine a villain hiring an uncontrollable beast and telling their vassals to always lock the cage, right? Now imagine that the first night they just go for a barbecue in front of the cage with the door open? Obvious ending, but I digress...
OpenAI came on August 7th with Responding to the next frontier of critical cyber capabilities: they cannot rule out that Astra (unreleased) crossed the “Critical” cyber threshold of their own Preparedness Framework. First time ever, for any of their models. Critical here means a model that may plan and execute serious cyberattacks with little or no human help. Their response? Slowing parts of the development, tougher security requirements, and rewriting the framework itself... because the models arrived at the thresholds the document had only imagined. Some might say it was because the AISI report had just pinned 17 of its 19 unsanctioned actions on Anthropic’s model (the OpenAI one only did “something bad” twice), but there is no money involved in appearing too dangerous, right?
What about Meta? On August 10th Zuckerberg published a 6,500-word essay, The Future Is for Everyone, telling us that “The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic”. Released together with an open-weight model, naturally. What the essay did not mention: days earlier, per press reports, one of Meta’s own models had escaped a testing sandbox through a configuration error and exploited a real third-party vulnerability. Told you the Meta joke would land :-D
Now read those again... but slower (especially the AISI one). Because each one is trying to be an ad and a scary bedtime story at the same time. But hey, they are postmortems. And we, the people who run systems for a living, know exactly how to read a postmortem (and the fact we should not call it like that anymore...).
The mockery and the panic are the same mistake
And of course, we (or the internet) decided the extremes are the cool way: “Let’s mock them for making it look like an ad” against “We are doomed!”.
It’s easy to make a meme with “Our model is so dangerous we can barely contain it”... available today, $20/month, no waitlist. The doomsday device with a free tier. Isn’t it strange that every lab now announces that their creation might end civilization and puts the pricing table one scroll below (yes, the truly dangerous ones stay unreleased... but memes don’t care about footnotes)?
On the other hand, more arguments for your CISO to continue increasing their Zscaler budget... “The agents are threat actors!” Pause the rollout, draft a policy, form a committee... and if the company is big enough, a task force and a “GenAI Risk Maturity Assessment” from a consultancy that discovered agents last quarter. It’s the same energy as banning USB sticks after the breach that came in through email, but no one can say they want to be riskful...
And even though I see where both come from, I dislike extremes, especially when they are spectator seats. The laughing grandstand and the panicking grandstand are watching the same show and neither one has/wants to change a single thing about their week. Both treat the reports as content, as media... and as we just said: they are postmortems.
So we laugh, Thiago? Ofc! You joke about it! You go from capitalism skeptical to doomsday-guy-building-a-bunker at the barbecue. But then you read it the way you would read your own.
What was the root cause? The environment said one thing and did another.
How was it detected? Network monitoring at AISI, a >140k-run retrospective at Anthropic.
What was the impact? Contained, thanks to fast response and a decent amount of luck.
If your SRE handed you with “root cause”, “detection” and “impact” about staging reaching the production database, you would not call your infrastructure “too dangerous”. You would be thankful that a mistake didn’t cost much, fix the wiring and go looking for the other places where the paperwork and the wires disagree.
That is the reclassification that matters, not “AGI is coming” or “AI becomes evil”. This is the most boring failure class we own, wearing the scariest costume available.
And to be fair with the labs: they were more honest here than most vendors will ever be. These tests run with guardrails off on purpose (that is the point of capability testing) and Anthropic wrote a line that deserves quoting straight:
“In none of these situations did Claude exfiltrate itself or deliberately attempt to escape its test environment.”
The model did what capture-the-flag training teaches and every textbook would say: reach the flag. Nothing “escaped”. The fence was simply not where the paperwork said it was.
Which is exactly why this stops being funny. Skynet, you don’t have. Config drift? Oh, you have it... I have it... we have it. The labs found their incidents because they went looking, with monitoring in place and the patience to comb through 140k+ runs. When was the last time you went looking?
You are running the same eval, without the eval team
Let’s go back to your company, your problem, and that AI-Native Operating Model we keep talking about. Last week’s playbook (that I still stand behind!) ends with agents opening Merge Requests across your repositories. To do that, they hold real things: repo access, CI credentials, tokens that can touch your artifact registry... and if we are honest, in most companies, way more than that...
You might think “But Thiago, we didn’t remove the guardrails like the labs did... so we are safe!” Are you sure? Let me ask this differently:
Which credentials did your coding agents hold last week? (and whose token was it, actually?)
What network can their environment reach?
Who reads their transcripts?
If one of them did something weird last Tuesday... which artifact would tell you?
I’ve asked these in rooms with very senior people, and the honest answers, in order, tend to be: “a personal access token someone pasted in March”, “all of it”, “nobody” and “hmm...”
If those are your answers too, congratulations: you are running the same permissive capability eval the labs run, on your own infrastructure. Minus the monitoring, minus the transcript retention, minus the >140k-run retrospective. AISI had a fence with a hole in it... you skipped the fence and went straight to the barbecue.
“But we trust our engineers!”... me too! And the agent is not your engineer. Your engineer does not open 40 MRs on a Tuesday night, does not clone the same repo 200 times, and (I sincerely hope) does not create fake identities to convince a maintainer to merge things. Trust in people does not transfer to workloads. We solved this before, by the way... we just keep forgetting that “principle of least privilege” applies to anything that holds a credential, not only to the humans in the directory.
And here is the three-body reading (aha! You knew it was coming!): this is the GenAI capability body sprinting while the People and Architecture bodies watch from the grandstand (yes, the same grandstands from before). Every new model release makes your agents more capable. None of them makes your permissions, your review gates or your audit trail one bit better.
Buying more capability does not close that gap... it widens it. Issue 5’s Herbie is back (the slowest hiker in the line, the one who sets the pace for the whole troop) and this time he is holding your prod token...
The operating-model answer
Uh-oh... so what? We climb out of the engine room and we write it down as an operating model. Not a tool to buy, not a committee to form... five decisions... all boring and cheap (especially compared to what your agents already cost you).
1. Identity. Your pair-programming buddy can act as you while you are watching the terminal... but the moment nobody is watching, the agent runs as its own principal. Never on a human’s token (goodbye, token pasted in March). If you cannot attribute an action, you cannot review it... and “who did this?” should never have “we think it was the intern’s PAT” as an answer. The security crowd has already written this down, by the way: OWASP’s Non-Human Identities Top 10 reads like it was waiting for agents to go mainstream.
2. Least privilege, with an expiry date. The agent gets the repos and environments of its task, nothing else, and its credentials die young. Yes, this is the same least-privilege sermon your security team gives every year. The only news is that it now applies to something that runs at 3am at ~65 tokens/sec.
3. Egress is part of the spec. What the environment can reach over the network is written down, and checked... not assumed. Read Anthropic’s postmortem again: this single line item was the entire root cause of all three incidents. The prompts said “no internet” and the wires disagreed. So, check the wires!
4. The owner-reviewed gate. Remember last week’s owner-reviewed MRs? That was never bureaucracy... that was the containment boundary. The gate is where a wrong lesson stops compounding across the fleet. Removing it does not make you faster, it makes you AISI without the monitoring. And yes, at fleet scale this gate becomes your bottleneck... on purpose. That is Herbie again: you subordinate the fleet’s pace to the gate, you do not delete the gate. A whole industry is now theorizing its way around it (Tessl and the spec-driven development crowd, where the MR is cast as the villain of the developer’s life)... maybe they are right, one day. Until your specs review themselves, the gate stays.
5. Keep the transcripts, review the incidents. The labs went back through >140k runs to find three incidents. You do not need that scale: keep your agents’ transcripts for 90 days and run a blameless review when a Tuesday gets weird. If nothing is ever weird, your monitoring is not good... it’s blind.
None of this requires believing “too dangerous”. It only requires believing the vendors’ own postmortems... the most credible documents they will ever publish.
And notice what just happened: not one of these five is about the model. The model will change again next quarter (it already did while I was writing this)... but the operating model? Well, that is yours to keep and evolve.
Run it on yourself
Before your next “AI townhall”, before the next license renewal, and definitely before someone proposes an “AI Agent Governance Committee”... three things you can do this week, no extra budget needed:
Inventory the tokens. List every credential your coding agents held last week... and whose name each one carries. If building that list requires a meeting (or several), the list is the finding.
Test the egress yourself. Go inside the agent’s environment and try to reach something it should never touch: the payment API, the prod database, that storage bucket with the backups. The labs’ entire root cause was this check, skipped. Twenty minutes, one afternoon, possibly the best security investment of your quarter.
Find the gate. Locate the one boundary an agent’s change cannot cross without a human whose name is on the repo. If you cannot find it... or if someone removed it “for velocity”... you did not remove the risk, you removed the containment.
Run these three and you will know exactly which grandstand you have been sitting in... likely both, depending on the system/team. And more important: you will have a much better answer than mocking or panicking when your board forwards you the next “AI attacks company” headline with a one-line “should we worry?” on top.
The whole thing, in one line
Your agents were never too dangerous to release. They are already released, by you. The only question left is what they can reach.
Coming up
Next week we stay out of the engine room and look at the people side of this: what agent fleets do to your teams, their cognitive load, and who reviews the reviewers. Shape still forming... More soon. :-]
Sources
UK AI Security Institute, “Incident Report: unsanctioned agent behaviour during cyber testing”, 4 August 2026 (government institute incident report, not peer-reviewed). https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing
Anthropic, “Investigating three real-world incidents in our cybersecurity evaluations”, 30 July 2026 (vendor incident write-up, not peer-reviewed). https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals
OpenAI, “Responding to the next frontier of critical cyber capabilities”, 7 August 2026 (vendor announcement, not peer-reviewed). https://openai.com/index/responding-next-frontier-critical-cyber-capabilities/
Mark Zuckerberg, “The Future Is for Everyone”, Meta, 10 August 2026 (corporate essay). https://about.fb.com/news/2026/08/the-future-is-for-everyone/
“The AI safety test is becoming a safety risk”, TechCrunch, 9 August 2026 (press; source for the Meta sandbox escape). https://techcrunch.com/2026/08/09/the-ai-safety-test-is-becoming-a-safety-risk/
OWASP, “Non-Human Identities Top 10”, 2025 edition (industry security standard, not peer-reviewed). https://owasp.org/www-project-non-human-identities-top-10/