fix(native): retire a finished per-turn owner past its own quarantined interrupted bootstrap (#20140)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A follow-up turn on a per-turn session must retire the previous
turn's runner state before it starts a new runner. Retirement proves the
previous owner finished, and it also proves that every quarantined copy
of that session root is settled
> - If restart recovery interrupts a run's own start (for example a hard
kill during `session.open`), recovery proves the half-started bootstrap
held no provider work, quarantines that root, and bootstraps the same
run again. The quarantined copy keeps the run's identity
> - After that run finished, retirement checked the old bootstrap copy
as if it were a finished owner. An unfinished `session.open` (or an
opened session without a turn) is never "settled", so the follow-up
failed with `runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate` and needed a Retry
> - This pull request skips a quarantined copy of the same owner when
the same bootstrap proof that let recovery quarantine it still holds
> - The benefit is that a follow-up after a restart-interrupted start
continues on its own

## Linked Issues or Issue Description

Refs #20137

**What happened?**
Local restart-chaos K13 regression storm w-20261108T235957-s9401 (master
4420a5adcf): 102 of 105 tasks passed. Two of the three failures share
this cause.

- **t58 (codex C1):**
- A K1 at 00:15:42 interrupted run `78dd97f1` during its first
`session.open`. Recovery quarantined that root (`run.prepare` completed,
`session.open` pending, no events) and bootstrapped again.
- The run then completed. A second K1 hit 190 ms after its result was
accepted, and recovery finalized the result.
- Follow-up `def668a6` failed with `runner_state_identity_mismatch:
prior_owner_terminal_state_indeterminate;
retirement_refused=quarantine_journal_commands_unsettled(session.open.pending)`.
- **t67 (claude C1):** the same pattern. The quarantined copy of run
`990936c0` holds a completed `session.open` with only session-open
events and no turn, so the refusal was
`quarantine_journal_terminal_unsettled`.

Both copies are in the kept instance's quarantine directory under the
previous run's own identity. Each Retry succeeded.

**Expected behavior**
A quarantined copy of the finished owner's own earlier bootstrap that
provably held no provider work does not hold the follow-up. Retirement
proceeds and the follow-up starts without a Retry.

**Steps to reproduce**
1. Let a per-turn run's first bootstrap be quarantined after a restart
(the run bootstraps again and finishes).
2. Start a follow-up turn.
3. Before this change, retirement refuses with
`quarantine_journal_commands_unsettled(session.open.pending)` or
`quarantine_journal_terminal_unsettled`.

This is not a code regression. The quarantine-copy check dates from
#19061. Today's recovery fixes let more interrupted starts bootstrap
again within the same run instead of blocking, and each such restart
leaves this kind of copy behind.

**Paperclip version or commit**
4420a5adcf (wave). Branch rebased on current master.

## What Changed

- `native-session-executor.ts`: the quarantine loop in
`inspectFinishedPerTurnRunnerdOwner` skips a copy when both of these
hold:
  - its durable identity matches the finished owner exactly;
- `runnerdStateProvesIncompleteBootstrap` holds for it. That is the
proof restart recovery uses before it quarantines an interrupted
bootstrap: no provider turn, and for a local runner, its journal and
processes proven gone.
  
  Every other copy still has to pass the settled-projection proof.
- `native-recovered-owner-followup.test.ts`: a red/green test covering
both cases:
  - an interrupted bootstrap copy is retired past;
  - a copy that had already issued a turn still holds the follow-up.

## Verification

- Red/green: on master, the new admitted case fails with exactly the t58
error. With the change it is admitted. The turn-issued copy is refused
on both.
- `vitest run` of `native-recovered-owner-followup.test.ts`,
`native-session-executor.test.ts` and
`native-runner-restart-recovery.integration.test.ts`: 3 files, 1504
tests passed (14 skipped).
- `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` (`server`): passed.

## Risks

- Low risk. The skip reuses the existing bootstrap-incomplete proof and
requires an exact identity match, so a foreign or work-holding copy
still blocks. Nothing is deleted; the copy stays in quarantine.
- Not covered: t22 in the same storm, a follow-up whose own unconnected
`run.attach` root was hit by a K1 (`authority_indeterminate`), has a
different shape and is tracked separately.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use
(shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-08 19:21:32 -06:00
1 parent 546cfee919
commit bd97a3e6f7
2 files changed
+71 -1

No files matched your search

@@ -5612,6 +5612,7 @@ async function inspectFinishedPerTurnRunnerdOwner(input: {
if (!isSafeNativeStateDirectory(quarantine)) return refuse("quarantine_unsafe");
for (const name of readdirSync(quarantine)) {
if (!name.startsWith(`${basename(input.root)}.`)) continue;
if (quarantinedBootstrapOfOwnerHeldNoProviderWork(resolve(quarantine, name), identity, localRunner)) continue;
const copy = readSettledRunnerdProjection(resolve(quarantine, name), identity, { localRunner });
if (copy.refused !== undefined) return refuse(`quarantine_${copy.refused}`);
}
@@ -5627,6 +5628,15 @@ async function inspectFinishedPerTurnRunnerdOwner(input: {
return { finished: true, run: prior.run, recheck };
}
export function quarantinedBootstrapOfOwnerHeldNoProviderWork(root: string, identity: Record<string, string>,
localRunner: boolean): boolean {
if (!isSafeNativeStateDirectory(root)) return false;
const copied = readRunnerdDurableIdentity(root);
if (!copied || Object.entries(identity).some(([key, value]) => copied[key] !== value)) return false;
return runnerdStateProvesIncompleteBootstrap(root, { localProcesses: localRunner })
|| (!localRunner && runnerdStateProvesNoRemoteProviderTurn(root));
}
async function retireFinishedPerTurnRunnerdOwner(input: Parameters<typeof inspectFinishedPerTurnRunnerdOwner>[0])
: Promise<{ retired: true } | { retired: false; refused: string }> {
const proof = await inspectFinishedPerTurnRunnerdOwner(input);