Following an issue to an autonomous merge

On 2 August 2026 at 00:11, my test script creates an issue on a brand-new GitHub repo. At 00:12, a pull request opens. At 00:13, it is merged. At 00:15, Release Please publishes version 1.0.0. Nobody clicked anything in between. The repo is public and everything is still there.

What I want to describe here is not the speed. It is what happens during those four minutes, and above all what stops the loop from merging just anything. Shipyard is the framework I built to have Claude Code agents deliver issues. This post follows one issue end to end, then looks at the guardrails and the limits.

The principle: the script decides, the agent executes

With agent-setup, my previous tool, I started every sprint and every workflow myself, in a session where the model chained the steps on its own. To deliver without me, I needed a sequence that did not depend on what the model chose to do next. So in shipyard, control flow is a bash script, agent-loop.sh, and each agent runs one bounded step in a fresh claude -p session.

The agent drives nothing. It returns text with a few markers the script can read: COMMIT: for the commit message, PR-BODY for the PR description, NEEDS-HUMAN: when it is stuck, VERDICT: for the review. Anything that changes state on GitHub, a label, a merge or a close, is done by the script. Memory lives in git, on the board and in .project/state.json. Never in a model’s context.

Read the diagram top to bottom. Two things matter. The orchestrator talks to every actor, but the actors never talk to each other. And the reviewer only comes in after CI: no green, no judgement.

Step 1: the issue is the contract

The test issue asks for a status.json file, linked from the home page. It has four acceptance criteria as checkboxes and a definition of done: node --test and CI green, no added dependency. That is the only spec the agents get. A vague issue gives a vague PR, and no amount of guardrails fixes that.

Step 2: the dev session

The loop takes the next “Todo” issue in the GitHub Project’s current iteration. It moves it to “In Progress”, creates an isolated worktree on the feat/2 branch and starts Sonnet with the issue, the project context and the engineering rules for its role. Test first, implementation second.

test('status.json is valid JSON reporting ok status', () => {
  const status = JSON.parse(readFileSync('public/status.json', 'utf8'));
  assert.deepEqual(status, { status: 'ok' });
});

test('the Pages app links to the machine-readable status page', () => {
  const html = readFileSync('public/index.html', 'utf8');
  assert.match(html, /<a[^>]+href="status\.json"[^>]*>Service status<\/a>/);
});

Before pushing anything, the script itself replays the project’s lint, tests and build, using the commands stored in state.json. If a gate fails, a fix session gets the exact command and the tail of its output, with a budget of two attempts. A badly formatted commit message is normalised to Conventional Commits rather than rejected, and the log says so. Then come the push, gh pr create and the move to “In Review”.

The dev phase does not wait for CI. It moves on to the next issue. Review is a separate phase of the same run, which picks up every “In Review” issue.

Step 3: CI first

The review phase starts by waiting for the checks. It polls gh pr checks and waits while they are not registered yet, then while they run. There are three possible outcomes. Green: on to the judgement. Red: the reviewer is skipped, and the fix session gets the failing job’s name and the tail of its log. No answer before the timeout: that is a third state, retried without spending a round, because a stuck runner is not a bug in the code.

In this test, the build job passed in eight seconds. On repos scaffolded by shipyard, branch protection requires three checks, build, trivy and pr-title (commitlint), in strict mode. So the branch must be up to date with its base.

Step 4: a review in a fresh session

On green CI, the script starts a new session, Opus by default. It gets the issue context, the PR diff and nothing else. It did not watch the code being written and does not know what the developer “meant”. Its prompt says what it does not judge, since CI covers that, and what it does judge: is each criterion met by the diff? Were tests weakened to pass? Does the diff go out of scope? Does coverage of new code reach the threshold?

Commence ta réponse par **exactement une** ligne :

VERDICT: approve
ou
VERDICT: request-changes

Si `approve` : arrête-toi là. Rien à ajouter, pas de résumé de ce qui va bien.

Si `request-changes` : **au plus 3 points bloquants**, le plus grave d'abord :

**1. <le défaut en une phrase>**
- Constat : <fichier:ligne> — <ce que fait le code, factuel>
- Conséquence : <ce qui casse, pour qui, dans quel cas>
- Attendu : <l'action précise à faire>

The output format is strict, and that is not cosmetic. Early reviews ran three screens long, and the next fix tackled the least important point. Now it is at most three blocking points, each with file and line, within a 2,000-character budget. Any verdict other than approve is posted as a PR comment, so a human coming in later can see the discussion. It is then fed back into a fix session on the same branch.

Step 5: the merge, and what follows

On approve, the script rebase-merges under a global lock. Two PRs merging into the same base at once used to fight over “base out of date”. If the merge fails, the script first checks with the server that the PR was not merged already, then looks for a conflict to resolve. After that it closes the issue, sets the board to “Done”, updates the vault and notifies my phone.

The rest is standard GitHub tooling set up by the bootstrap: the develop → main promotion, then Release Please, which reads the Conventional commit and opens the release PR. For the test issue, less than four minutes passed between its creation and 1.0.0.

When it does not go through

The happy path is the shortest to write. Most of the script handles everything else. Every loop is bounded: 3 dev attempts, 4 review rounds, 2 base re-integrations on conflict. Past that, the issue goes to “Blocked” and the loop posts a comment with the cause and the action to take.

This screenshot comes from a run of the same test half an hour earlier, with the review budget set to two rounds. The comment is clear and gives the action to take. But look at the timeline: GitHub had merged the PR 24 seconds before the escalation. On the server side, the merge went through. On the script side, the command had returned an error, and the loop carried on as if nothing had been merged. Since then, every merge attempt is checked against the PR’s actual state before being called a failure. That is what lets the next run pass.

It is not the only case. On one of my projects, an issue was escalated 54 seconds after a commit that addressed all three points of the previous review: the last fix was never re-reviewed. Now it is, before any escalation.

Escalating fast has a cost: blockers go stale. The PR you were waiting on gets merged, or the human answers in a comment. So every run starts with a triage phase. An Opus session re-reads each blocked issue against the repo’s actual state and returns one decision: send it back to dev, send it to review, close it as a duplicate, wait for a dependency, or a human really is needed. That session cannot touch GitHub: gh is removed at the CLI level. The script applies the decision.

The guardrails, as a list

• Local gates before the push, required CI before the judgement. A model’s judgement is never the only gate.
• Review in a fresh session, on a different model from the dev, with an explicit checklist against weakened tests.
• Branch protection on develop: build, trivy and pr-title required, up-to-date branch enforced.
• Tools removed at the CLI level, not just discouraged in the prompt.
• Per-issue budgets persisted: a review that survives a crash resumes its counter instead of getting a fresh one.
• Monorepo scope: with a pkg:* label, a diff touching another package is rejected before the PR.

The limits, honestly

This end-to-end test is real, but small. The issue is synthetic, and the reviewer ran on Haiku there to save quota. The dependency PR is simulated by the test account, because GitHub does not let a test impersonate Dependabot. The day I re-ran the smoke test for this post, one check out of 171 was red.

• The reviewer can be wrong. It judges criteria, it proves nothing. If the issue is badly written, it will approve a PR that follows it to the letter.
• Three sources of truth. state.json, the labels and the GitHub board drifted apart. An issue whose work was already pushed went back to dev, produced an empty diff and got escalated, over and over. I stacked up reconciliations, but the real fix was a single source. That is what koklo-cli does.
• Sequential. Review and merges go one at a time. That is safe, but slow on a big sprint.
• Tied to GitHub and to a plan. No remote CI, no merge. And on a Claude plan, the loop runs in bursts: it stops when the quota runs out, then resumes when it comes back.
• Branch protection is not enforced for admins, and the loop runs with my token. The guardrail is that the script never calls gh pr merge --admin. That is a code convention, not a server-side constraint.

Wrapping up

Four minutes from an issue to a release is a demo. What I care about is a loop that stops at the right moment, with one clear sentence and a command to run. That is what lets me start a sprint from my phone, then look only at what really needs me.