Files
PaperClipAI/server
DottaandClaude Opus 5.5 196108acb9 fix(native): bootstrap again when a relaunched remote runner exited before provider work (#19951)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Restart recovery decides whether an interrupted native run can
resume, bootstrap again, or must wait for a person.
> - A remote run is blocked as "ambiguous" when recovery cannot prove
what reached the sandbox runner.
> - In a Daytona chaos wave (#19947), two runs were blocked even though
the controller had recorded that the relaunched runner exited before any
provider work.
> - This pull request accepts that recorded runner exit as proof and
bootstraps the run again, under the existing attempt budget.
> - The benefit is that a restart no longer turns a run with no provider
work into a failed, blocked run.

## Linked Issues or Issue Description

Refs #19947 (Daytona data), #19831 (durable-ownership plan), #19934 (the
stuck claimed wake from the same runs).

**What happened?**
Wave `w-20261108T012041-s9301`, runs `450f277d…` and `9b93d40c…`, went
through this sequence:
1. Restart recovery proved the remote bootstrap incomplete and
relaunched it.
2. The new runner exited: "runnerd exited while waiting for
run.prepare". The executor settled `bootstrap_retry` with
`providerSessionEstablished: false`, `providerEventsExist: false` and
`checkpointExists: false`.
3. The next restart recovery saw the recorded runner PID and blocked the
run as `ambiguous_remote_recovery_evidence`.

**What did you expect?**
Bootstrap again, because the runner is proven gone and no provider work
happened.

**Steps to reproduce**
1. `cd server && npx vitest run
src/services/native-runtime/native-runner-restart-recovery.integration.test.ts
-t "relaunched remote runner"` fails on master with `blocked /
ambiguous_remote_recovery_evidence`.

## What Changed

- `native-restart-recovery.ts`: new proof
`remoteRunnerExitedBeforeProviderWork`. It applies to a remote sandbox
with a prepared workspace and a settled `bootstrap_retry` failure whose
original code is `native_runner_process_exited`, with no provider
session, events or checkpoint and no lease or successor. Such a run is
classified as `bootstrap_incomplete`
(`remote_runner_exited_before_provider_work`). It passes the same retry
admission check as `remote_runner_never_launched`, and the same
three-attempt budget.
- Tests: the s9301-shaped retry plus five counter-cases.

## Verification

- `cd server && npx vitest run
src/services/native-runtime/native-runner-restart-recovery.integration.test.ts
-t "exited remote runner|relaunched remote runner|remote bootstrap
retry|never requested a runner|exact-checkpoint retry"`: 14 pass.
- Red/green: the new bootstrap test and the budget counter-case failed
on master (both returned `blocked / ambiguous_remote_recovery_evidence`,
the same as the s9301 rows) and pass here. The other four counter-cases
(not a runner exit, session established, provider events, checkpoint)
block on both.
- Loop: focused vitest with one embedded Postgres, 26–50 s.
- Not run locally: the server typecheck and the full suite (memory
limits on this host). Required CI covers them.

## Risks

- Fail-open risk: the proof needs the executor's own settled record of a
runner exit with no provider work, plus a fresh check that no checkpoint
or provider evidence exists. A recorded PID without the exit record
still blocks, as before.
- Previous-release story (rules/upgrade-safe-run-state.mdc): no
persisted shape changes. Rows without these failure fields keep today's
outcome.
- Nothing runs before `turn.start`. This is restart recovery only.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with
tool use (shell, file edits).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 21:08:35 -06:00
..