Shipyard

The screenshots on this page come from throwaway public repos that my end-to-end tests create on GitHub, such as shipyard-e2e-full-cycle. The issues are synthetic, but everything else is real: real Claude Code sessions, real GitHub Actions CI, real merges.

How it works

1. An issue with checkable criteria

Everything starts from an issue. The bootstrap creates it from the spec, or I write it by hand. It has a goal, acceptance criteria as checkboxes and a definition of done. The dev session gets nothing but that text, the project context and the engineering rules for its role.

2. The loop develops and opens the PR

agent-loop.sh takes the next “Todo” issue in the current iteration, creates an isolated worktree on feat/<n> and starts a fresh claude -p session on Sonnet. The test comes first, then the implementation. Before any push, the script itself replays the project’s lint, tests and build. If a gate is red, a fix session gets the exact command and the tail of its output. Then the script commits in Conventional Commits, pushes and opens the PR. The issue moves to “In Review”. The dev phase does not wait for CI: it moves on to the next issue.

3. The hard gate, then the judgement

The review phase picks up every “In Review” issue. It first waits for the required checks to register and finish. A red CI goes straight back as fix feedback, with the failing job’s name and the tail of its log. The reviewer is only called on green CI, in a fresh Opus session that never saw the code being written.

4. Merge, release, promotion

On approve, the script rebase-merges under a global lock, closes the issue, updates the board and the vault, then notifies my phone. In the full-cycle test, the same repo then goes through a dependency bump, the promotion to main and the Release Please release. The dependency PR is simulated by the test account there, because GitHub does not let a test impersonate Dependabot.

5. When a human is needed

An issue with contradictory criteria must not produce code. The dev session returns NEEDS-HUMAN:, the loop moves the issue to “Blocked” and posts a single comment: the cause, and what to do to resume. On the next run, a triage phase re-reads every blocker against the repo’s actual state. Anything that was only blocked for technical reasons goes back into the queue.

At the end of a sprint, a retrospective is opened as an issue, facts first.

Under the hood

The model never decides what happens next. Control flow is a 2,800-line bash script. Each agent runs one bounded step in a fresh session and returns output that the script parses: COMMIT:, NEEDS-HUMAN:, VERDICT:. Progress lives in git, on the GitHub board and in state.json, never in a model’s context.

Read the diagram left to right. Every arrow is a status change written by the script, never by the agent. All loops are bounded: 3 dev attempts by default, 4 review rounds, 2 base re-integrations on conflict. Past that, the issue goes to “Blocked”. Getting out of “Blocked” is the triage phase’s job. An Opus session judges the blocker there, and the script applies its decision. For that session, gh is removed at the CLI level with --disallowedTools.

while [ "$round" -le "$MAX_REVIEW" ]; do
    round=$((round + 1))
    _review_state_set "$n" "$round" "$conflict_fixes"

    # Hard gate: objective signals. Waits for checks to register + complete.
    local ci_rc=0
    ci_is_green "$pr" || ci_rc=$?
    # … rc 2 = timeout : on réattend sans consommer de tour
    if [ "$ci_rc" -ne 0 ]; then
        verdict="request-changes"
        review_out="$(ci_failure_report "$pr")"
    else
        review_prompt="$(render_prompt "$LOOP_DIR/prompt-review.md" "$review_ctx" qa-reviewer)"
        review_out="$(claude_invoke "$SHIPYARD_REVIEW_MODEL" "$review_prompt")" || break
        verdict="$(grep -m1 '^VERDICT:' <<<"$review_out" | sed 's/^VERDICT:[[:space:]]*//')"
    fi

    [ "$verdict" = "approve" ] || post_review_comment "$pr" "$round" "$MAX_REVIEW" "$verdict" "$review_out"

    if [ "$verdict" = "approve" ]; then
        _with_lock "$REPO_ROOT/.project/.merge.lock" merge_pr "$pr" || { …; }
        finalize_merged_issue "$n" "$pr" "$SHIPYARD_REVIEW_LABEL" "$round"
        return 0
    fi
    # sinon : session de correction (Sonnet) avec le retour réinjecté, push, tour suivant
done

This is the core of the merge gate. CI is checked before the reviewer, and any verdict other than approve is posted as a PR comment, so verdicts stay readable afterwards. The merge runs under a lock, because two PRs merging at once used to fight over “base out of date”.

What the machine says about itself

The host check, run on my machine:

The loop smoke test stubs gh and claude but pushes to a real local git remote. It replays 29 scenarios in about a minute, with no quota. The day I ran it for this page, it returned 170 green checks and one red: in one scenario, the loop’s own doc pass still tripped the monorepo scope check. I am leaving it as it is.

Where it comes from, where it goes

shipyard reuses agent-setup’s foundation: shell renderer, templates, TDD roles and state.json. It drops Codex to focus on Claude Code and adds the autonomous loop. koklo-cli then takes over the loop, and above all the order of its guards, but rewrites it in Rust on a single SQLite database.

What I take away

• The script decides, the agent executes. The model never picks the next step. A failure then becomes a state of the script, which a smoke test can replay, rather than a decision you have to guess at afterwards.
• Every incident becomes a guardrail, and a test. A CI timeout counted as a failure, a last fix never reviewed before escalation, a review budget reset on every resume: about thirty cases are encoded in the code, each with its comment and its smoke scenario.
• Three sources of truth is two too many. state.json, GitHub labels and the board drifted apart. An issue whose work was already pushed went back to dev, produced an empty diff, then got escalated, over and over. That is the main reason for the koklo-cli rewrite.