The Agent Stopped, the Loop Kept Going

Before, an agent that stopped left the gate red and the loop ran the task again; after, the agent writes a file the loop checks before the gate, and the loop stops and asks me

I leave coding agents working on their own inside a loop: loop-until-green, a Bash script that runs Claude Code headless (claude -p) on one task, pass after pass, until a command I choose exits 0.

Some tasks hold a decision that isn’t the agent’s to make: a schema, the shape of a public API, which of two behaviours people expect. For those, the prompt told the agent to stop and explain. It took a review of the script to notice that this stopped the agent, but not the loop.

How the loop decides

The agent writing the code never decides that the work is done. Two checks do, and a ledger carries memory from one pass to the next:

A gate. One shell command that exits 0 only when the work is done, such as npm test && npm run build. The agent is never asked whether it’s finished, only to make the command pass.
A judge. When the gate goes green, a second model reads the task and the diff and answers PASS or FAIL: did the writer weaken the tests or hardcode the answer? It didn’t write the code, so it has nothing to defend.
A ledger. Each pass starts with an empty context, so .loop/LEDGER.md records what every pass changed and why it failed, and the last failure goes into the next prompt.

After each pass the script checks for an escalation first, then the gate, then the judge; a red gate or a rejected diff goes back to the writer through the ledger

That’s the loop as it is now. The first check, for an escalation, wasn’t there.

Why “stop” didn’t stop it

The script chose what to do next from the gate’s exit code, never from what the agent said. An agent that rightly stopped left the code as it was, so the gate stayed red. To the script, that was an ordinary failure. It logged it, put the gate’s error output into the next prompt and ran the same task again, up to the pass limit, eight by default.

So the next pass got a red gate and the same instruction: make it green. Its prompt carried the gate’s error output, not the agent’s reasoning about the decision. Every pass after that was another chance for the decision to be guessed, with nobody asked. The pass limit capped the cost, not the outcome: the run could end with the question stepped around.

I don’t know how often this happened: the review found it in the code, not in a run. But it could have happened on any task that held a call meant for me.

A file the loop checks first

The fix gives stopping a mechanism instead of an instruction. The agent writes its reasoning and the options it sees to .loop/ESCALATED.md, and leaves the code as it is. After every pass, the script looks for that file before it runs the gate:

if [ -f "$ESCALATION" ]; then
  echo "✋ writer escalated at iter $i — loop stopped. This is your call: $ESCALATION"
  exit 3
fi
if gate; then …

Three details matter:

It runs before the gate. Checked after it, an escalation that happened to leave the gate green would exit 0 straight past the open question.
It has its own exit code. 3 means “your call”, apart from 0 (done) and 1 (out of passes), so whatever runs the loop can tell them apart.
It’s cleared at the start of each run, so an old question can’t halt a new run.

I tested it with a fake claude that escalates: with the gate red, the run stops after the first pass with exit 3; with the gate green, it still exits 3, not 0; and a file left over from an earlier run doesn’t stop the next one. What the test can’t show is whether a real agent writes the file when it should. That’s down to the prompt, so the instruction is in both the loop’s prompt and the task file, which lists the decisions under “Escalate to the human — STOP, don’t guess”.

The same rule runs through the whole loop

An instruction to the agent is a request. Anything that has to hold needs a check outside the agent:

“Are you done?” became a gate: a command that passes or doesn’t.
“Don’t touch the tests” is still in the prompt, but the agent runs with Bash and permissions skipped, so nothing stops it. The judge reading the diff is what catches it.
“Stop and ask” became a file the loop looks for.

The checks need checking too. My terminal app wraps claude and prints an escape sequence when a session starts. Captured by $(…), that sequence landed in front of the judge’s verdict, so the first line no longer began with PASS, and a real pass read as a rejection. The script now strips escape codes before it reads the verdict.

Checking the script again for this post turned up two more. The judge was given the task file’s path, not its text, and one turn to answer: in seven runs out of seven it spent the turn trying to open the file, and the CLI printed “Error: Reached max turns (1)” instead of a verdict, which the script read as a rejection. And the verdict match accepted PASS at the start of any line, in any case, so a FAIL whose reason began “Passes only because…” counted as a pass. The judge now gets the task inline with its tools turned off, and only its first line counts.

What it doesn’t do yet

It has no spending cap. It stops after eight passes by default, each capped at 80 turns, but never at a cost.
The judge fails open: if it returns nothing, the script prints a warning and accepts the gate alone.
The judge reads the first 12,000 bytes of git diff, which leaves out new files git doesn’t track yet and anything the agent staged or committed.
It only works where a machine can say done. “Is this copy good?” has no gate, so a loop would grade itself and pass. I do those by hand.

Try it

The whole script is below, 112 lines of Bash. Save it as loop-until-green somewhere on your PATH, make it executable, and run it from the root of a git repo with Claude Code installed. Put TASK.md in a .loop/ folder that git ignores, so the ledger never ends up in a pull request. In it, describe the task and list the decisions the agent must hand back to you under a heading such as “Escalate to the human”. The agent runs with permissions skipped, so run the loop in a container or a VM you don’t mind losing.

#!/usr/bin/env bash
# ─────────────────────────────────────────────────────────────────────────────
# loop-until-green — reusable closed-loop harness (Reflexion-style, any project).
#
#   Usage:  loop-until-green <task-file> <gate-command> [max-iters]
#
#     <task-file>     a spec the agent implements (convention: .loop/TASK.md)
#     <gate-command>  a shell command that exits 0 IFF the work is DONE — this is
#                     your EXTERNAL VERIFIER, e.g.  "cargo test -p foo && cargo build"
#                     or  "npm test"  or  "pytest -q"  or  "./verify.sh".
#     [max-iters]     hard stop (default 8).
#
#   Exits 0 verified · 1 out of passes · 2 bad usage · 3 escalated: your call.
#
#   writer(claude -p, reads LEDGER) → ESCALATED? → GATE(your command) → judge(haiku)
#     → done | loop | escalated: stop and hand you the call (exit 3)
#
#   Context resets each pass; MEMORY persists on disk (that's what makes reset safe):
#     • git working tree        = the artifact (episodic memory of the CODE state)
#     • <task-dir>/LEDGER.md     = attempts + WHY they failed (reasoning memory)
#     • CLAUDE.md                = durable cross-run lessons
#
#   Run it from your repo root. Put TASK.md + LEDGER.md under .loop/ (gitignored)
#   so the harness never leaks into a PR. Verify `claude --help` flags per version.
# ─────────────────────────────────────────────────────────────────────────────
set -uo pipefail
TASK="${1:-}"; GATE_CMD="${2:-}"; MAX="${3:-8}"
[ -n "$TASK" ] && [ -n "$GATE_CMD" ] || { echo "usage: loop-until-green <task-file> <gate-command> [max-iters]" >&2; exit 2; }
[ -f "$TASK" ] || { echo "task file not found: $TASK" >&2; exit 2; }
TASK_ABS="$(cd "$(dirname "$TASK")" && pwd)/$(basename "$TASK")"
LEDGER="$(dirname "$TASK_ABS")/LEDGER.md"
[ -f "$LEDGER" ] || printf '# LEDGER — attempts + why they failed (read me first)\n\n' >"$LEDGER"
# The writer's stop signal. "STOP and explain" only stops the WRITER — without a
# sentinel the harness sees an unchanged tree as an ordinary RED and re-prompts
# the same task, so the call meant for a human gets quietly guessed on a later
# iteration. This file is how an escalation reaches the loop.
ESCALATION="$(dirname "$TASK_ABS")/ESCALATED.md"

gate_out=""
gate() { gate_out="$( bash -c "$GATE_CMD" 2>&1 )"; return $?; }

fail_excerpt() {                       # pull the signal out of the gate's noise
  local e
  e="$(printf '%s' "$1" | grep -E 'error(\[|:)|no field|unknown|cannot find|FAILED|failed|panicked|Error|✗|not ok|AssertionError' | head -20)"
  if [ -n "$e" ]; then printf '%s' "$e"; else printf '%s' "$1" | tail -c 1500; fi
}

strip_ansi() {                         # de-noise the judge's stdout before matching
  # A wrapped `claude` (a terminal app's shell shim, say) may print an OSC session-announce
  # escape to stdout; under judge="$(claude -p …)" it's CAPTURED and prepended to the verdict,
  # defeating the ^PASS match (a real PASS reads as REJECT). Strip OSC (ESC ] … BEL|ST)
  # + CSI (ESC [ … final).
  LC_ALL=C sed -E $'s/\x1b\\][^\x1b\x07]*(\x07|\x1b\\\\)//g; s/\x1b\\[[0-9;?]*[A-Za-z]//g'
}

echo "loop-until-green · task=$TASK_ABS · gate=[$GATE_CMD] · max=$MAX"
# One run = one delimited block of N iterations in the ledger (runs stack below).
printf '\n═══════════ run · %s · max=%s ═══════════\n' "$(date '+%F %H:%M')" "$MAX" >>"$LEDGER"
rm -f "$ESCALATION"                    # a previous run's escalation must not halt this one
last_fail=""
for i in $(seq 1 "$MAX"); do
  printf '\n════════ iter %s/%s ════════\n' "$i" "$MAX"

  prompt="Read $TASK_ABS, then read $LEDGER — it records what earlier iterations tried and why each failed. Do NOT repeat a dead end. Implement the task so this gate command exits 0:  $GATE_CMD  . Do NOT modify the tests / the verifier itself. If the task lists an escalate decision that you hit, do NOT guess and do NOT work around it: write your reasoning and the options you see to $ESCALATION, leave the tree as it is, and stop. That file halts the loop and hands the decision to the human — it is the only way out of here that isn't a guess."
  if [ -n "$last_fail" ]; then
    prompt="$prompt"$'\n\n── The previous iteration ended RED. Gate output: ──\n'"$last_fail"$'\n── Diagnose THIS specifically; continue from the current tree, do not start over. ──'
  fi
  prompt="$prompt"$'\n\nBefore finishing, append to '"$LEDGER"$' a "## iter '"$i"$'" heading then 2-4 lines: what you changed, and what still fails and why (if known). Bank durable lessons into CLAUDE.md.'

  claude -p "$prompt" --allowedTools "Read,Edit,Write,Bash,Grep,Glob" --max-turns 80 --dangerously-skip-permissions

  # Checked BEFORE the gate: an escalation is a human's call even if the gate
  # happens to be green, and a green gate would otherwise exit 0 around it.
  if [ -f "$ESCALATION" ]; then
    printf '## iter %s · ESCALATED\n%s\n\n' "$i" "$(cat "$ESCALATION")" >>"$LEDGER"
    echo "── escalation ──"; cat "$ESCALATION"
    echo "✋ writer escalated at iter $i — loop stopped. This is your call: $ESCALATION"
    exit 3
  fi

  if gate; then
    diff="$(git diff 2>/dev/null | head -c 12000)"
    # The spec goes in inline and tools are off: given one turn, a judge that opens a
    # file spends it there and prints "Error: Reached max turns (1)", never a verdict.
    judge="$(claude -p "Task spec:
$(cat "$TASK_ABS")

DIFF:
$diff

Did this meet the task WITHOUT gaming it (no edited/weakened verifier, no hardcoded values)? First line exactly PASS or FAIL, then one line why." --model haiku --tools "" --strict-mcp-config --max-turns 1 2>/dev/null)"
    judge="$(printf '%s' "$judge" | strip_ansi)"   # strip wrapper OSC 697 before the verdict match
    printf '## iter %s · gate GREEN\n%s\n\n' "$i" "${judge:-<judge unavailable>}" >>"$LEDGER"
    if [ -z "$judge" ]; then
      echo "⚠ gate GREEN; judge unavailable — accepting the deterministic gate."
      echo "✓ VERIFIED at iter $i (gate only).  Review: git diff"; exit 0
    fi
    echo "── judge ──"; echo "$judge"
    # Only the first non-blank line is the verdict: "FAIL / Passes only because…" is a FAIL.
    if printf '%s\n' "$judge" | grep -m1 '[^[:space:]]' | grep -qE '^[[:space:]]*PASS([^[:alnum:]_]|$)'; then
      echo "✓ VERIFIED at iter $i — gate green + judge PASS.  Review: git diff"; exit 0
    fi
    last_fail="Gate was GREEN but the judge REJECTED the diff (likely gamed): $judge"
    echo "judge flagged gaming → reason fed forward → loop"
  else
    last_fail="$(fail_excerpt "$gate_out")"
    printf '## iter %s · gate RED\n```\n%s\n```\n\n' "$i" "$last_fail" >>"$LEDGER"
    echo "✗ gate red → reason logged to $LEDGER + fed into next prompt → loop"
  fi
done
echo "✗ hit cap ($MAX) without a verified pass — INSPECT $LEDGER, don't just re-run."
exit 1