From Vibe Coding to Supervised Autonomy

The compromise everyone landed on—let the agents write, keep humans reviewing, and simply leave the rest of the process alone—looks sensible enough. In the consults we run, that buys customers somewhere in the range of a 20-30% gain before the number stops moving. Because every hour an agent saves shows up in front of someone who has to absorb it, so throughput ends up governed by how quickly your engineers can take in what the agents produce.

That’s not even the hardest problem. The bigger challenge is that this arrangement is less safe than it looks. "Someone reviewed it" is a claim nobody can really check afterward. Even when it does happen, reviewers skim as volume climbs. And in the age of agents, that volume only keeps climbing.

As long as confidence comes from a person having looked, speed and safety pull against each other.

Confidence built from evidence behaves differently. Criteria agreed before an agent starts, checks that run the same way on every change, and signals from production after release— none of that is bounded by anyone's attention, and all of it leaves something you can go back to.

In this article, we propose a five-level maturity model for agent-driven software delivery. The model measures how a team produces confidence in a change and how early it does so.

Each level names the thing that produces confidence.

At Level 0: Vibe Check, the software appears to work and that's the whole guarantee. Level 1: Borrowed Speed, rests on a person's word after reading the change. Level 2: Durable Intent, adds criteria agreed before the agent starts, so the question of whether a change is correct has an answer that predates the code. At Level 3: Earned Trust, automated checks produce that answer on every change and a human reviews the exceptions. Level 4: Directed Outcomes, pushes verification into production, where staged rollout and monitoring carry the load and people spend their time on intent and policy.

The levels aren't a queue you join at one end. A healthy organization runs Level 0 spikes and Level 3 production services in the same sprint, placing each workstream where its risk warrants.

Several models already circulate in this space, and each answers a different question than this one.

Google's DORA research, drawing on nearly 5,000 technology professionals, found that AI acts as an amplifier. It magnifies whatever your delivery system already is, including the parts that were broken before anyone installed an agent. Foundations decide your return, which is worth knowing and still leaves you guessing at where you currently stand.

Birgitta Böckeler at Thoughtworks gave the industry its working vocabulary for the middle of this range, splitting spec-driven development into spec-first, spec-anchored, and spec-as-source. The taxonomy is precise about the artifact and quiet about the organization around it. Addy Osmani drew the boundary between vibe coding and AI-assisted engineering, which is the distinction most teams are reaching for when they describe what they do.

The newest entries come from the tooling vendors. Factory's Autonomy Maturity Model scores repositories against eight technical pillars, running from linting and build systems through to observability and governance. Its signals are binary by design, which makes the model practical to run across a large estate. What it grades is your codebase. A repository can clear all eight pillars while the team working in it produces no verifiable confidence, because no pillar asks who wrote the tests or whether anyone agreed with the criteria.

Readiness is a property of your code, and a script can grade it. What we're describing is a property of your approach, and seeing it means looking at how the work actually happens.

Nothing below Level 1 belongs in front of a customer. If code will handle a transaction or come near payment and personal data, someone on your team has read it and can explain it. That's the minimum defensible position, and we treat Level 1 as the floor for that reason.

Almost every organization that reaches the floor then stops on it, and the economics explain why. Generating a change costs a fraction of what it did; understanding one costs what it always did. CodeRabbit, which sells AI code review and so has a stake in the answer, found AI-co-authored pull requests carrying 10.83 review issues each against 6.45 for human-only work across 470 PRs. GitClear, looking at 211 million changed lines, found code churn and duplication rising over the same period that AI authorship spread. Reviewers are handling more changes and finding more in each one.

The floor also erodes under load. A merge approval records that someone looked, without recording what they checked. As volume rises the gap between those two widens, which is a real change in your risk profile and one that will not appear in any report you receive.

Level 0: Vibe Check

Confidence comes from watching the thing run. The safety mechanism is disposability: spikes, prototypes, internal tools, a working proof of an idea before anyone commits budget to it. That value is real, and treating it as shameful is how teams burn weeks over-engineering work meant to be thrown away. The failure is Level 0 output crossing into a system that users and production data depend on.

Level 1: Borrowed Speed

A person reads the change and vouches for it, everything else about the process is unchanged from before agents arrived. That means throughput is capped at reading speed, and the guarantee lives in someone's memory, where nobody else can inspect it.

The speed at this level is borrowed from the AI coding harness and underlying models. Nothing about how your teams work has changed, so the gain doesn't compound and it's hard to defend the token economics in a finance conversation.

It's also borrowed against your reviewers. Every extra change an agent produces draws on a fixed line of human attention, and the interest is paid in queue time and shallower reads. That kind of borrowed capacity gets called in at the worst moment: during a release crunch, over a holiday, or when the one person who understands a service moves on.

Level 2: Durable Intent

Acceptance criteria are written and agreed before generation begins, and they outlive the session. Review stops being a reconstruction exercise, because the reviewer checks a change against something specific and no longer has to infer what was wanted. An agent can draft those criteria. Ratifying them is the part that stays human, and it's cheap, which is what makes this level worth reaching first.

What this level asks for is discipline; the tooling is incidental. Teams that stall usually do so because criteria get written after the fact to match what the agent produced.

Level 3: Earned Trust

Automated checks produce the answer before anyone looks: types, tests derived from the agreed criteria, contract tests across service boundaries, dependency and secret scanning, and hard limits on what a change can touch and spend. Each change arrives with evidence—what was claimed, what was proven—and humans spend their review time on exceptions.

This is the first level where adding agents increases throughput, since checks scale with volume while your senior engineers' capacity doesn't. It's also the level nobody can sell you: Level 3 is continuous delivery with the gate pointed at an agent, so a team that couldn't run CD two years ago can't run Level 3 today.

Distributed estates raise the bar: a single change can cross four or five services owned by different teams, plus an integration nobody has touched in years. Coverage means gating all of them, not just the repo where the work started.

Level 4, Directed Outcomes

Verification extends past release. Staged rollout, monitoring on behavior the business cares about, and automatic rollback catch the class of problem that nothing before deployment can find. People set intent and policy, and agents work toward outcomes inside that envelope. This is supervised autonomy. The supervision is the part that makes it work, because a human remains accountable for what gets built and why.

Few organizations run this way for the systems their customers depend on. Many demonstrations of Level 4 turn out to be Level 3 with good stage presence, which is worth knowing before you plan a roadmap around it.

Here’s the thing: a check doesn't get tired. It applies the same way to the hundredth change as the first, at 3 a.m., nine weeks into a release freeze. A reviewer at volume can't do that, and asking them to is how the floor erodes.

Evidence compounds the effect. When a change carries a record of what was verified, that record is still inspectable months later during an incident or an audit. The same machinery bounds where an agent can reach, caps what it can spend, and logs what it touched, which turns agent access from an open question into a governed one.

Confidence only counts when it comes from something independent of what produced the work. If one agent writes the specification, the code, and the tests in a single pass, the tests assert what the code does. Green means those three artifacts agree with each other, which is a much weaker claim than it appears. A team in that position has the ceremony of Level 3 without the substance, and that's more dangerous than Level 0, because it manufactures assurance where Level 0 at least admits to having none.

Independence has to hold in a few places. Criteria are fixed before generation and don't get revised to match the diff. The thing that checks the work doesn't share context with the thing that produced it. And the gate can't move in the same unreviewed change as the code, because the cheapest route to green is editing the gate.

The model has a limit worth stating plainly. Every level above zero raises confidence that the output matches the intent, and none of them interrogates the intent itself. An agent will build the wrong thing quickly, with full coverage and a clean audit trail. Judgment about what to build gets more valuable as execution gets cheaper.

Ask an engineer one question: the last time an agent produced something wrong, what caught it? The answer reveals where your team sits.

You, eventually, is Level 0. You, in review, is Level 1. A test written from criteria agreed beforehand is Level 2. An automated gate, before anyone looked, is Level 3. A canary and a rollback is Level 4.

Then ask who wrote that test, and whether they had the implementation in front of them. That follow-up moves a lot of teams down a level.

Run this per workstream, and note that differing responses per workstream doesn’t mean something is wrong. The answer for an internal reporting tool and the answer for the service that moves money should differ, and if they don't, one of them is wrong.

Decide where vibe coding earns its keep. Level 0 is fast and it's the right answer for a class of work: throwaway prototypes, internal experiments, proving an idea before it gets funded. Write down which work qualifies, and write down the trigger that forces it up a level, which is usually the moment anyone outside the team starts to depend on it. Organizations that leave this undefined either ban Level 0 outright and lose the speed, or let Level 0 work drift into production one exception at a time.

Write down where the floor sits. Most organizations already have a review policy. Few have one that says anything about work an agent produced, so the question falls to individual habit under deadline. Name the systems where nothing merges without a person who can explain it, and make it explicit that the rule applies whoever or whatever wrote the code.

Move one workstream up a level this quarter. Pick a team, pick the level above where they sit today, and change one thing. For most teams that means criteria agreed and recorded before the agent starts, with tests written from those criteria and not from the finished implementation. Compare the rework rate against a similar piece of work done the usual way. Keep it to one experiment.

Measure what share of merged change clears an automated gate. Count what proportion of what shipped last quarter was verified by something other than a person under deadline pressure. Repository coverage is the wrong denominator, because a pipeline that exists and a pipeline that gates are different things. The number you get is your real Level 3 coverage, and in the estates we assess it sits below what the org chart implies.

Ten times the throughput lives above the level where a person is the checkpoint. Getting there is unglamorous work on process, and it's available to any team willing to start with one workstream and one quarter.

That's the work we do with engineering teams, and you can read more about how we approach it on our agent-driven development page. If you'd like help placing your teams against these levels, or deciding which workstream to move first, get in touch with our team.