Files
PaperClipAI/server
DottaandClaude Opus 5.5 85bb942aad fix(native): prove an unconnected bootstrap from the sandbox's own Runner state on a volume-free server (#20992)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs execute through a Runner (`paperclip-runnerd`) in a
remote sandbox. When a Runner exits before `session.open`, the run
restarts its bootstrap on the same run.
> - #20972 lets host preparation restart that bootstrap when the Runner
wrote `runner-state.json` but no provider state. It needs a proof that
this run's own interrupted Runner is stopped or gone.
> - That proof is recorded only when the server still has the run's
local Runner controller root. A volume-free server, or a root that held
no identity, skips it. The live chaos wave after #20972 still failed
with `runner_harness_state_mismatch: resumed_unavailable`.
> - This pull request lets host preparation establish the same proof
from durable sandbox evidence when no local proof exists.
> - The benefit is that the restart also works on volume-free servers,
which is how Cloud runs.

## Linked Issues or Issue Description

**What happened?**
Restart-chaos wave `w-20261112T002811-s6402` ran #20972 (Claude through
ACPX, Daytona warm, `--ephemeral-server`, K1/K2/K15 with
`--k15-fresh-home`, seed 6402, `--runner-bind-fault`). 5 of 6 tasks
needed Retry, with 6 `resumed_unavailable` failures. Run `fbb1666d` in
t01: the sandbox held `runner-state.json` for this run with `lifecycle:
connecting`, the same Runner instance and session,
`lastConnectionProtocolVersion: null`, and no provider state. That is
exactly the state #20972 restarts, but its proof was absent.

**Root cause**
`recordRemoteBootstrapRestartProof` is called only inside the
`existsSync(scoped)` block of the restart path in
`server/src/services/native-runtime/native-session-executor.ts`, after
the local controller identity is read. With no local root (volume-free,
a fresh home, or an emptied root), `provenOwnBootstrapRestart` stays
false, so `inspectOwnUnpreparedBootstrap` from #20972 returns nothing
and the resumed branch throws `resumed_unavailable`.

**Expected behavior**
A run with a `bootstrap_incomplete` recovery claim whose sandbox Runner
state proves it never connected restarts its bootstrap without Retry.

## What Changed

- `inspectOwnUnpreparedBootstrap` now runs for any
`bootstrap_incomplete` claim of this run. When no proof was recorded, it
records one only if the sandbox Runner state proves it never connected
(`remoteRunnerStateNeverConnected`: `lastConnectionProtocolVersion:
null`, `lastControllerCommandSeq: 0`, `reconnectCount: 0`, empty outbox
and processed commands, no pending terminal delivery). The exact Runner
process must also be stopped or proven absent by the existing
`stopStaleRemoteRunnerBeforeBootstrap`, which uses the Runner state's
own identity. All earlier conditions from #20972 still apply: same run,
Runner instance and session, a lifecycle other than `suspended`, no
failover backup and no archived rotation snapshot.

## Verification

- Red on master (#20972 merged): `pnpm --filter @paperclipai/server exec
vitest run src/services/native-runtime/native-session-executor.test.ts
-t "bootstraps only an untouched provider session"`. The new
`interrupted_bootstrap_unprepared_unrecorded` cells failed: resumed
lease `Error: runner_harness_state_mismatch: resumed_unavailable`,
created lease no restart.
- Green: those cells pass. `_connected` (a Runner state that connected,
no local proof) and `_unproven` (the stop is not proven) keep the
refusal. The full file passes (1409 of 1409). Server `tsc --noEmit`
passes.
- Chaos, same cell with `--runner-bind-fault`. Wave
`w-20261112T002811-s6402` (#20972 only): 1 of 6 passed, 6
`resumed_unavailable`. Wave `w-20261112T010800-s6402` (this change): 3
of 6 passed, 2 `resumed_unavailable`. In this wave, run `0a080e83`
restarted its bootstrap three times on the same sandbox: three
`.bootstrap-restart-*` roots were moved aside. The baseline wave on
master before #20972 (`w-20261111T230537-s6402`, no injected fault)
passed 2 of 6.

## Risks

- The new path runs only for a `bootstrap_incomplete` claim of this run
when the full harness check fails, so the normal path is unchanged and
adds no wait before `turn.start`.
- A Runner state that ever connected, received a command or holds outbox
data keeps today's refusal. The moved-aside root is kept for inspection.
- Not fixed here, and seen in both waves: every Runner on a sandbox
listens on one fixed port (43127 in all captures). A live Runner of
another run or session on the same sandbox therefore makes each relaunch
fail with `runner_ingress_bind_conflict`, until the bounded retry is
exhausted (`native_session_retry_exhausted`). A later attempt can still
end with `resumed_unavailable` or `resumed_not_suspended`. That needs
its own fix in predecessor retirement or port selection.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edit, test runs). Default context window,
extended reasoning on.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Refs #20972, #20928, #20953, #20863.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 20:07:02 -06:00
..