mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 16:02:36 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs prepare a remote workspace, then start a runner and a provider session > - A graceful server shutdown closes the HTTP listener before it drains runs > - A run that is still preparing then asks for the local runtime API origin, and the lookup fails with `local_runtime_api_listener_unavailable` > - Recovery did not know this code. It treated the failure as an ambiguous provider state and blocked the task behind a board-owned action, although no runner or provider process existed > - This pull request classifies the closed listener as a definite pre-session failure > - The benefit is that a deploy no longer strands a preparing task. The same run gets a bounded bootstrap retry that the next server process claims ## Linked Issues or Issue Description **What happened** A native Codex task on a remote sandbox spent about ten minutes in workspace preparation. A deploy started a graceful shutdown during that time. When preparation finished, the executor called `resolveLocalRuntimeApiUrl()` to build the tool authority. The listener was already closed, so it threw `local_runtime_api_listener_unavailable`. No runner backend or provider session had started. The failure classifier returned the generic `native_session_interrupted` code. `nativeProviderRecoveryEvidence` then returned `ambiguous_state`. The run became a terminal failure, the task became blocked, and the recovery action told the board to resolve an ambiguous session and to not open a replacement session. The same signature occurred on three other runs during an earlier deploy. **Expected behavior** A failure that occurs before any runner or provider process exists is safe to retry on the same run. The task must stay in progress with a scheduled, bounded bootstrap retry. If a provider event or checkpoint exists, recovery must stay operator-owned. **Steps to reproduce** Start a native run. Close the server's HTTP listener before the executor creates the runner backend. Observe a `terminal_failure` with `recoveryMode: "ambiguous_state"` and a blocked task. The new integration test reproduces this with the real `resolveLocalRuntimeApiUrl()` and a real Postgres database. **Environment** Server native runtime, remote sandbox target, `codex_app_server` driver. Related recovery work: #19249, #19304, #19305. ## What Changed - `nativeSessionFailureSourceCode` returns a new `local_runtime_api_listener_unavailable` source code for this error. - `nativeProviderRecoveryEvidence` treats that code as definitely pre-session. The existing guard stays: the run must have no checkpoint and no durable provider event. - The chat failed-run retry accepts the code in its list of ordinary pre-session failures, so an exhausted startup retry can still be retried from chat. - New integration tests in `native-startup-retry.integration.test.ts` cover the retry path and the operator-owned path when a provider event exists. ## Verification - Red before the fix: the new test failed with `phase: "terminal_failure"`, `recoveryMode: "ambiguous_state"`, `recoverable: false` and a board-owned recovery. This matches the failed run on the deployed instance. - Green after the fix: `cd server && npx vitest run src/services/native-runtime/native-startup-retry.integration.test.ts` passes 26 of 26 tests. The retry test asserts `retryable_failure`, `bootstrap_retry`, backoff timing, an agent-owned `resume_native_run` wake, and a claim through `claimNativeSessionResumptions`. - Safety case: when a durable `turn.started` event exists, the same error stays `terminal_failure` with `ambiguous_state` and a blocked task. - `npx vitest run src/services/native-runtime/native-session-executor.test.ts -t "listener|classif|pre-session"` passes. - Not run: a live deploy that interrupts a preparing sandbox run. The test uses the real listener lookup and the real recovery tables, but not a real sandbox. ## Risks - Low risk. The change only affects runs that fail with this exact error and have no checkpoint and no provider event. - The retry uses the existing bootstrap budget of three attempts with backoff. Repeated failures still stop and hand the task to the board. - The GitHub callback bridge that started in the sandbox before the failure is a per-attempt helper. The existing cleanup path releases it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use, code execution and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>