Shift left assumes there is someone on your left

Two posts are worth reading back to back. Brian Houck at DX wrote what are code reviews even for?, arguing that AI did not break code review, it exposed habits that were already broken, and that automating review away would cost us the things review was quietly doing: knowledge transfer, shared ownership, architectural understanding. Rachel Laycock, CTO at Thoughtworks, replied with maybe we shouldn’t be reviewing all this code, agreeing about the value and disagreeing about the mechanism. Her question is the sharp one: why are we waiting until code review to do any of that?

They were on a panel together and they disagree in the useful way, where both positions are defensible. I read both and found I could not straightforwardly apply either, and it took me a while to work out why. It is not that either is wrong. It is that the room they are describing has other people in it.

The disagreement, stated fairly

Houck’s position is that the review queue is buckling and a large part of that is self inflicted. The research on what makes review work has been stable for a decade: small changes, a description that says why, automated checks before a human looks, reviewers who know the code. Most teams never did those things consistently, AI did not create that, it amplified it. Fix the basics, then use AI for risk stratification rather than blanket automation, and protect the thing review was actually producing. He puts that last part better than I could: “The visible output of code review is better code. The invisible output is a better engineering organization.”

Laycock’s position is that every item on that invisible list is better served earlier. Want to explore alternative solutions? Do it before one of them is implemented. Want knowledge transfer? Pair. Want juniors to see how experienced engineers think? Let them work alongside experienced engineers while the thinking is happening. Want architectural alignment? Design together, then encode the constraints as fitness functions. Keep human review for the exceptions: a fundamental architectural change, a sensitive security boundary, a huge blast radius, an unfamiliar part of a critical system, or somebody saying out loud that they are not confident.

I think Laycock has the better argument about mechanism and Houck has the better argument about what is at stake. That is not a fence sitting position, because they resolve into a single practical question, and it is the question neither of them has to answer at their scale: which changes are allowed to skip the conversation, and who decides?

Both prescriptions have a headcount in them

Read Laycock’s list again as a shopping list. Pairing. Mob programming. Team design sessions around a whiteboard. Collective ownership. Reviewing an architectural change as a team.

Every one of those requires a second person who knows enough about the system to be worth talking to.

I work one engagement at a time, usually as the only person touching a particular subsystem, often inside a product where the rest of the team has never opened the directory I am in. That is not an unusual arrangement. It is most contract engineering, most solo product work, and a large share of the small teams who actually own revenue critical code. When I built the commission and payout core inside Fluent Affiliate, there was no second person to pair with on the commission rules. There was me, the existing code, and users who would notice.

Houck’s prescription has the same shape from the other side. Protect reviewer time, avoid concentrating review on the same small group of experts, measure whether junior developers are learning. Those are org design instructions. They are good ones. They are also unavailable if the entire engineering organisation is three people and one of them is the founder.

So the honest translation problem is this. Laycock is right that waiting until a pull request is a bad time to have the important conversation. But if there is nobody to have it with, “shift left” does not move the conversation earlier, it deletes it. And Houck is right that the invisible output of review is shared understanding, but if there is nobody to share it with, review is not producing that either. Both of them are describing a loss I already took years ago.

What is left, once you subtract the second person, is narrower and more mechanical than either post suggests. It is worth naming precisely, because I think it is the part that generalises back up to larger teams.

Review by exception needs a written list of exceptions

Laycock names her exceptions in prose: architectural change, security boundary, huge blast radius, unfamiliar part of a critical system, low confidence. Every one of those is a judgement call made at the moment you are least able to make it, which is when you have just finished the change and want to be done.

Meta did not do it that way. The RADAR paper is the most concrete thing in this whole debate and it is worth reading past the headline numbers. RADAR is a multi stage funnel: it classifies each diff by authorship and source type, applies eligibility gates, static heuristics, a machine learned diff risk score, an LLM review, and deterministic validation, and only then lands the qualifying changes. Over 535,000 diffs reviewed, over 331,000 landed. The thing that made that work was not the LLM. It was everything in front of the LLM deciding what was allowed to skip a human.

You are not going to train a diff risk score, and you do not have the telemetry to calibrate one. But you do not need a model to answer the question the model is answering, because you already know the answer and have never written it down.

The classifier for a small team is a list of paths.

# review-policy.md
# Changes touching these paths do not land on the same day they are written.
# Everything else can go straight in.

app/Billing/**
app/Licensing/**
app/Subscriptions/**
**/*Webhook*
database/migrations/**   # additive only, see the migrations post
config/queue.php

That took me longer to format than to decide, which is the point. Most engineers I have worked with could write this list for their own system in an afternoon and have not, because it feels too obvious to be worth writing down. It is worth writing down for exactly one reason: written, it is a rule that applies when you are tired; unwritten, it is a judgement call that loses to wanting to be finished.

When there is no second reviewer, route to a procedure

Here is where the solo case stops being a smaller version of the team case and becomes a different problem.

On a team, an exception routes to a person. Laycock’s whole design depends on that: the exception list decides when a human looks, and there is a human to look. If you are the only person who understands the subsystem, that route does not exist. Escalating to yourself is not a control.

So the exception has to route to a procedure instead, and the procedure has to be something that makes a tired person skip it deliberately rather than by default. What I use is three questions, which are the same three I ask before writing billing code at all, answered in the pull request body rather than in my head:

# .github/workflows/review-policy.yml
name: review-policy
on: pull_request

jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Require the money-path answers
        env:
          BODY: ${{ github.event.pull_request.body }}
          # Keep this in step with review-policy.md. They will drift if you let them.
          PATHS: '^(app/(Billing|Licensing|Subscriptions)/|database/migrations/|config/queue\.php$)|Webhook'
        run: |
          changed=$(git diff --name-only "origin/${{ github.base_ref }}...HEAD")

          if ! printf '%s\n' "$changed" | grep -Eq "$PATHS"; then
            echo "No listed path touched."
            exit 0
          fi

          printf '%s\n' "$changed" | grep -E "$PATHS"

          fail=0
          for q in "Runs twice:" "Source of truth:" "Reconstructed by:"; do
            printf '%s' "$BODY" | grep -qF "$q" || { echo "Missing: $q"; fail=1; }
          done
          exit "$fail"

The check is dumb on purpose. It does not read the answers, it cannot tell a good one from a lie, and a determined person defeats it by typing three headings and nothing after them. I am not trying to catch a determined person. I am trying to catch myself at eleven at night, which is a much easier target and the one that actually costs money.

Two honest limits before anyone copies this. It runs on pull requests, so it does nothing about a push straight to the base branch, and a failing check blocks nothing unless you make it a required status under branch protection. Both of those are one setting each, and skipping them turns the gate into decoration. If you are the sort of solo developer who commits to main all day, the gate is the branch protection, and the workflow is just the thing that gives it something to fail on.

The property I care about is that it fires on the path, not on the person. It does not care whether the change was written by me, by a contractor, or by an agent, which matters because the standard should not change because the author is a model.

That is a deliberate divergence from Meta, and it is worth being straight about. RADAR routes by authorship first: its eligibility tree branches on authorship type before anything else, and bot authored and human authored diffs run through structurally different pipelines. I am doing the opposite, because the reason Meta can key on authorship is that it knows which automation produced each diff and has per source policy for each one. I do not have that, and neither do you. What I have is a directory layout that already encodes what the code can cost me if it is wrong.

One of those three prompts does most of the work, and it is the second delivery question. With at least once delivery, a webhook arriving twice is normal operation rather than an error case, so a handler that is only correct once is a handler that is wrong and has not been caught yet. That single question is most of what I have ever found in the payments edge cases that page you at 2am.

Blast radius is a property of your deploy, not of your diff

Laycock lists “a change with a huge blast radius” as an exception, which is right, and treats it as a property of the change, which is where it stops being right for the systems I work on.

Ship a WordPress plugin through wordpress.org and the release reaches every site with auto updates on, over the following day or so as each site’s update cron fires, on infrastructure you have never seen, alongside plugins you have never heard of. You do not choose who gets it and you cannot take it back.

The channel is less blunt than it used to be. WordPress.org shipped phased releases in August 2025, and the one strategy currently offered is worth turning on: opt in at release time and automatic updates are suppressed for 24 hours, so the first sites to take the update are the ones whose owners clicked the button and are therefore paying attention. That is a real control and I would use it on anything with an install base.

It is also not a canary, and the gap matters. It is opt in per release rather than a default. There is no percentage granularity, which is listed as a possible future iteration rather than something you can use today. It only reaches WordPress 6.6 and later. And third party update tooling has to honour the flag, which the announcement itself says is unknown. What you get is a 24 hour head start with your most engaged users, which is a good early warning system and not a way to limit how many people the change eventually reaches.

That still inverts the exception rule. On a big install base, “huge blast radius” is not the exception Laycock reserves human attention for, it is the ambient condition, and the exceptions are the few changes that cannot reach a user: a build script, a test fixture, an internal screen nobody outside the team opens.

So you stage the behaviour rather than the download. That is what feature flags inside an old plugin are for, and it is why migrations on a live install base stay additive. It is also a real cost that neither post accounts for, because at Meta or Thoughtworks the rollout mechanism is assumed to exist. When the channel cannot give you a percentage, a chunk of Laycock’s “review by exception” saving is spent back on flags and compatibility work. Worth it, and not free.

What is actually lost, and it is not knowledge transfer

Houck’s strongest paragraph is about what disappears quietly. His argument is that automating the review of a diff may successfully review that diff while transferring no knowledge, building no shared ownership, and surfacing none of the design rationale somebody will need half a year later. He borrows Margaret-Anne Storey’s framing of cognitive and intent debt for it, and I used the same framing when I wrote about why cognitive debt costs more in code that takes money.

The RADAR authors raise the same risk about their own system, though more mildly than the summaries of it suggest. Their line is that the trade off is currently favourable, because RADAR reviewed diffs are qualitatively different from the ones humans review anyway, but that the balance may shift as automation coverage expands. Houck reads that as something organisations should actively monitor, which I think is the right reading, and it is his reading rather than a sentence in the paper. Either way it is a candid thing to put in a paper reporting a production incident rate a fiftieth that of the comparison group.

But on a small team, knowledge transfer is not what is being lost, because it was not happening. What is being lost is narrower and more recoverable: the written reason.

A code review conversation is a terrible place to store a reason, and it happens to be where most reasons end up. It is off to the side of the code, keyed to a diff rather than to a line, and effectively invisible to whoever opens the file in eighteen months. That reader is going to be a stranger, or an agent, or me having forgotten. Asked why a guard clause exists, none of the three can answer from the code, and a model will produce a confident answer whether or not it knows.

So the substitute for knowledge transfer is not a ceremony, it is an artefact. Record the why next to the thing, at the moment you decide it, in a comment or a commit message that survives. A minute at the time, unrecoverable later. This is the one habit I would keep if I had to drop everything else on this page, and it is also the strongest argument for Laycock’s position that I can make from where I sit: if the reasoning is written down where it lives, the pull request was never the thing carrying it.

One number in this debate that I would not repeat

If you quote anything from the RADAR paper, do not quote this one. The abstract says RADAR “reduces median time to close by over 330%”.

A reduction of more than 100% is not a thing. You cannot remove more than all of the time, so whatever was measured, it was not a reduction. Houck reads it as a 3.3 times speedup, roughly a 70% reduction, and a commenter on his post flagged the same wording. That reading is the most likely one and it is still a reconstruction: read the same slip the other way round, as new time being 330% faster than old, and you get about 4.3 times and a 77% reduction instead. The paper publishes no raw medians to settle it.

I am pointing at this for one reason. It is the most quotable sentence in the abstract, which makes it the sentence most likely to end up in a slide deck arguing for something expensive, and nobody who repeats it will know which of the two numbers they are repeating.

The paper’s other results do not have that problem and are the ones worth carrying. A revert rate one third that of non RADAR diffs, and a production incident rate one fiftieth. Read those as what they are: an observational comparison between two populations the authors themselves describe as qualitatively different, since RADAR only ever handled the diffs its eligibility gates let through. That is not “automation made things fifty times safer”. It is evidence that a conservative classifier picks safe changes, which is an argument for stratification rather than for automation.

Where this leaves the argument

I think Laycock wins the mechanism argument and it does not help me much. Feedback is worth more closer to the decision, that is true whether or not there is anyone else in the room, and the version available to one person is not pairing. It is writing down what you expect the change to look like before you let anything generate it, and finding out immediately whether you were calibrated or just agreeable.

I think Houck wins the stakes argument, and the part that survives the translation to a small team is not the mentoring or the collective ownership. It is his narrower claim that a review culture which was already thin does not get better under more volume. Mine was thin. AI did not change what I should have been doing, it removed the constraint that used to limit how much unreviewed code I could produce in a day.

And I think both of them, plus Meta, converge on the same conclusion by different routes: the decision that matters is not how to review, it is what is allowed through without one. Meta answers it with eligibility gates and a machine learned risk score. Laycock answers it with team judgement in the moment. If you are one person, or three, you answer it with a list of paths you wrote down on a calm afternoon, and a check that fails the build when you touch them without doing the work.

FAQ