The bottleneck had shifted
For 20 years, engineering organisations had been structured around a single scarce resource: the production of correct code. Sprint planning, hiring loops, reviews, and career ladders were tuned to optimize the flow of an expensive operation. It had become inexpensive. What is now expensive is determining what needs to be built, ensuring that what has been built conforms to requirements, and maintaining the system’s understandability for those responsible for it.
The evidence supports this organisational rather than tooling framing. In DORA's State of AI-Assisted Software Development 2025 – nearly 5,000 respondents; about 90% AI use – AI is an amplifier; it amplifies both the advantages of healthy organisations and the pathologies of unhealthy ones. Productivity increased, as did instability. GitClear's 2026 maintainability study of 600 million+ commits is similarly revealing: duplicate blocks 81% higher than 2023, error-masking constructs 47% higher, and "moved" lines – the signature of refactoring – have dropped from 21% to 3.8% of changed lines between 2022 and 2026. These are structural measures, not defect rates, and their interpretation is disputed. But the trend follows logically from generation outpacing understanding: code is appended and not refactored, because nobody can refactor what nobody understands.
Combine that with a calibration issue. A randomised controlled trial by METR found experienced maintainers spent 19% more time working with early 2025 AI tools but perceived themselves to be 20% faster. METR has since redesigned the study, so the result should not be taken at face value. The perception disparity should. An organisation navigating AI strategy by developer satisfaction polls flies blind into the wind.
Draw the line at the oracle, not at difficulty
The useful distinction is not between “hard versus easy” work but whether a cheap, machine-checkable oracle exists. AI reliably handles mechanical transformation under a verifier: boilerplate, schema-driven clients, compiler-guided migrations, test scaffolds, documentation reconciliation, “make this suite pass” loops.
It is unreliable wherever correctness is defined outside the repository. Whether a retry is safe depends on whether a downstream endpoint is idempotent — a fact living in a runbook or a colleague's memory. Whether an authorisation check belongs before or after a feature-flag evaluation depends on a threat model. A model produces fluent, plausible code for both, and is confidently wrong at a rate low enough to evade casual review and high enough to cause incidents.
Review becomes intent verification
Line-by-line reading does not survive contact with a 2,000-line generated diff. Reviewers skim, approve, and the ritual becomes theatre. Google's code review guidance already ranks design above style; AI-scale volume makes that ordering non-negotiable.
The practical change is that a pull request must carry its intent, not just its diff. Require a change contract: the problem, the invariants preserved, the failure modes considered, the blast radius, and the rollback path. Humans evaluate that artefact, then spot-check code where the contract is riskiest – authorisation boundaries, money movement, PII, concurrency, migrations. Style and mechanical correctness go to type systems, linters, and AI reviewers. Then tier the process: auto-merge for reversible low-risk change, single-reviewer for standard change, and adversarial multi-reviewer for anything irreversible. Human attention is the scarcest budget in engineering; spend it on the five percent of changes that are expensive to undo.
Testing under generated volume
Generated tests are the highest-risk artifact in the pipeline: a system that writes both implementation and tests can converge on a mutually consistent misunderstanding — a green build around wrong behavior. A defensible stack layers accordingly:
- Specification tests authored by humans from requirements, never generated alongside the implementation they judge.
- Property-based and metamorphic tests that assert invariants rather than examples — cheap for AI to expand, hard to satisfy by accident.
- Mutation testing as the meta-test: the only practical way to know whether an inflated suite detects anything at all.
- Consumer-driven contract tests at service boundaries, where AI-authored assumptions silently diverge.
- Evaluation harnesses for probabilistic components: golden sets, statistical acceptance thresholds, distributional regression checks, and the failure modes in the OWASP Top 10 for LLM Applications.
- Production verification — progressive delivery, shadow traffic, SLO-linked rollback — as the last honest oracle.
Architecture is now the leverage point
When implementation is cheap, a bad boundary is no longer amortised across months of labour; it materialises immediately as ten services replicating the same wrong assumption. Modularity acquires a second purpose beyond human cognition: it bounds the context an agent must reason over and the blast radius when the agent is wrong. Governance must therefore be executable rather than advisory — architecture fitness tests in CI, enforced dependency rules, policy-as-code, provenance metadata on generated code, machine-readable decision records. A constraint an agent cannot read is one it will violate.
A concrete operating model
Roles. Product engineers own intent-to-verified-change and orchestrate agents. Platform engineers own the paved road, agent harnesses, and CI — their product is the AI development environment itself. A small, decisive architectural function owns boundaries and irreversibility. Quality engineering shifts from executing tests to authoring specifications, properties, and evaluations. Security is embedded in the change-contract loop, not a release gate.
Topology. Team Topologies holds up well: stream-aligned teams with hard ownership boundaries, a platform team, and enabling teams that teach delegation and verification skills rather than syntax.
Metrics. Retire lines of code and PR counts; under AI they measure the machine, not the organisation. Track change failure rate, time to restore, rework within fourteen days, review latency, and one new measure worth instrumenting: comprehension coverage — the share of production-critical code with a named human who can explain its invariants. Balance it with the SPACE framework so throughput is not bought with unmeasured cognitive load.
Management. Hire for decomposition, verification, and debugging under uncertainty. Promote the quality of irreversible decisions, not the volume of authored code, and rebuild the apprenticeship path deliberately: juniors once learned judgment by writing the easy code, and that rung has been automated away. Accountability does not move — the engineer who merges owns the change completely, whoever wrote it. And because nothing in the agent loop spontaneously produces comprehension, schedule it: refactoring, deletion, and documentation reconciliation as first-class capacity. Birgitta Böckeler's field notes on generative AI in engineering remain a useful antidote to planning from vendor benchmarks.
The failure mode to design against
The organisation at risk is not the one that is adopting AI slowly. It is the one shipping more code than it can explain: every system has authors but no owners, tests pass because they were written to pass, and the first person to read a module end-to-end is an incident responder at 3 a.m. Speed without comprehension is not velocity. It is a deferred incident load, borrowed at an unpriced rate.
The design question for CTOs is therefore not how to get more code out the door. It is whether the organisation's capacity to understand and safely change its systems is growing at least as fast as the systems themselves. Control theory describes what happens otherwise: a loop slower than the process it governs does not stabilise the system. It oscillates.
