Write-Protected Acceptance Tests: Keeping Validation Apart From the Code an Agent Wrote

When a coding agent can edit both the implementation and the tests, a passing build proves only that the two agree with each other. If the agent misread the requirement, it can write code that is wrong and tests that confirm the wrong behaviour, and the pipeline goes green. The fix is keeping the acceptance tests in a place the agent cannot touch.

GitHub's documentation on code owners describes the mechanism that makes this possible. A CODEOWNERS file names who must approve changes to specific paths. Combined with branch protection, it lets you declare that certain files require a named human reviewer before any change to them merges, regardless of who or what opened the pull request.

Why a green build stops meaning what you think

A test suite is a claim about intended behaviour. When a person writes the test and a person writes the code, the test is an independent statement: it encodes what the author believed the system should do, written down before or alongside the implementation. A disagreement between the two shows up as a failing test, and someone investigates.

When one agent produces both, that independence is gone. The test now reflects the same reading of the requirement as the code. If that reading is wrong, both artefacts are wrong in the same direction, and nothing in the build disagrees. You have a suite that passes and a feature that does the wrong thing.

This is the default behaviour of any loop where the thing under test and the test for it share an author. The agent is optimising for a green build, and editing the test is a valid way to get one.

Three controls that keep validation apart from the code

The goal is a validation path that does not derive from the implementation the agent wrote. Three repository controls get you there, and none of them require a new tool.

• Put acceptance tests behind CODEOWNERS. List the acceptance test directory and name a human team as the required reviewer. Any pull request that touches those files now needs that team's approval, so an agent cannot quietly rewrite an assertion to match its own code.
• Protect the path. Branch protection on the main branch, set to require code owner review, enforces the rule at merge time rather than trusting convention. An agent-opened pull request that edits a protected test file cannot merge until the named human signs off.
• Add a per-release check that test files carry no agent edits. Before a release branch cuts, diff the acceptance test paths against the previous release and confirm every change there was authored and approved by a person. If the diff shows an agent commit in a protected path, the release stops until someone reviews it.

Each control is cheap. The first two are configuration files. The third is a few lines in the pipeline that compare a diff against a path list.

Where the human-authored tests should come from

Protecting the tests only helps if the tests say something independent of the code. The strongest version of this ties each acceptance check to the requirement rather than to observed behaviour. An acceptance test written from the requirement asks whether the system does what the specification demands. A test written after the fact, by reading the code, asks whether the system does what it already does, which is not a test at all.

So the sequence matters. Write the acceptance criteria from the requirement, as concrete examples, before the agent starts. Hold those examples in a protected path. Let the agent produce the implementation and whatever unit-level tests it likes inside the paths it owns. The acceptance layer stays outside its reach and judges the result against the requirement, not against the code.

This also sets a clean boundary for review. A reviewer looking at an agent-generated pull request can check that the implementation satisfies the protected acceptance tests and that no protected test changed. Two questions, both answerable from the diff.

What this does not fix

Write protection stops the agent from editing the check. It does not make the check correct. A human-authored acceptance test can still be wrong, miss a case or encode a stale requirement. The control buys you independence, not coverage.

It also does nothing for the behaviour an acceptance test never exercised. If your suite only checks the happy path, the agent can produce code that fails on inputs nobody wrote an example for, and the protected tests will pass. Keeping validation apart from the code is the first condition for trusting a green build, not the only one.

What to do this week

Start with one release journey. Pick the service where agent-assisted changes land most often, and do three things.

• Add a CODEOWNERS entry for its acceptance test directory, naming a human team, and turn on branch protection that requires code owner review on the main branch.
• List which existing tests are acceptance tests tied to a requirement and which were written after the code by reading it. Move the first group into the protected path. Treat the second group as implementation detail, not validation, until someone rewrites them from the requirement.
• Add a release-time step that diffs the protected paths and fails if any change there lacks a human author and approval.

By Friday you have a service where a green build means the implementation agrees with a check a person wrote and the agent could not alter. That is a narrower, more honest claim than most pipelines make today.

Anystack's pods work this way by default. Every change starts as a written spec, the acceptance checks come from the requirement rather than the implementation, and each change carries a named reviewer of record who neither wrote nor prompted it. The quality engineering work lands inside your own repositories and pipelines, using the controls you already have, so the validation path survives handover rather than living in a tool you rent.

*Anystack Engineering runs three-person engineering pods that deliver inside your repositories with a written spec for every change, a named reviewer of record and delivery measured per engagement. Engagements from £18,000 per month. How the pod delivers.*