mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 01:57:31 +01:00
fix(native): let a Stop before the provider session opens continue once (#19206)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A remote task restores its saved workspace into a sandbox before the agent starts. > - When that restore fails, the run records only `remote_workspace_operation_failed: operation=restore exit=1`. > - The run then fails as `setup_failed`. No retry starts, and a `blocked` task gets no recovery action. > - An operator cannot tell which step failed, and nobody owns the next step. > - This pull request makes the sandbox scripts name their failed step in a closed grammar, and it gives a restore failure a bounded retry or a named board recovery action. > - The benefit is a visible cause and a clear owner. Saved work stays protected. ## Linked Issues or Issue Description Refs #19152 and #19187. Those changes name copy-back (save) failures. Refs #18852 and #19195. Those changes added the bounded bootstrap retry that this change reuses. **What happened?** A board reply woke an agent on a `blocked` parent task. The run failed during setup 13 seconds later. The only run event was `remote_workspace_operation_failed: operation=restore exit=1`. The run had no workspace operation row. The task stayed `blocked`, and its recovery actions showed `active: null`. The same opaque family also appeared for workspace preparation (`exit=1` only). **Expected behavior** The run names which restore step failed, without command output or paths. A failure before provider work uses a bounded retry when a retry can help. Otherwise, and when the retry budget is spent, the board gets a named recovery action. The saved version is never changed or overwritten. **Steps to reproduce** 1. Save a remote workspace, then start a new run on a different allocation. 2. Put a folder at the published workspace path in the sandbox before restore. 3. Observe that the base code records only `operation=restore exit=1`, then `setup_failed`, with no recovery action for a `blocked` task. **Paperclip version or commit** Base: `87dfba6818`. **Deployment mode** Legacy runtime with sandbox (Daytona) remote workspace storage. ## What Changed - The prepare, repository, and restore sandbox scripts track their current stage. On failure they print `remote_workspace_<op>_failed: stage=<stage>` plus closed fields: `reason=` (the script's own snake_case refusal), `archive=` (the archive helper's fixed reason), `code=` (errno), `disk=full`, `git=` (a fixed label chosen from Git output), and the existing `exit`/`signal`/`program` fields. - The server accepts that text only when it matches a strict grammar for the same operation. Other output is dropped, as before. - Provider refusals during the restore upload and restore command are recorded by closed labels, like copy-back. - A failed saved-workspace restore raises `RemoteWorkspaceRestoreError` with the server step (`snapshot_lookup`, `snapshot_download`, `sandbox_restore`). It is retryable only when the owner re-reads the same writer, epoch, and saved version with no unsaved work, and the sandbox did not name a refusal reason. - Setup failure handling classifies a pre-dispatch legacy restore failure as `remote_workspace_restore_failed`. A retryable failure uses the existing bounded bootstrap retry. A non-retryable or exhausted failure records a board-owned recovery action with the redacted cause, when stranded-work recovery does not scan the task status (for example `blocked`). - Stranded-work recovery gets a named notice for this code. Generic continuation recovery does not grant it a second budget, and legacy reconciliation does not invent provider actions for it. - `doc/execution-semantics.md` documents the contract. ## Verification - Red: the new transport tests failed on the base code with `remote_workspace_operation_failed: operation=restore exit=1` (and the same for prepare and repository). The new completion tests failed because the run became `setup_failed` with no retry and no recovery action. - Green: `npx vitest run src/services/remote-workspace-transport.test.ts src/services/heartbeat/run-completion.test.ts src/services/remote-workspaces.integration.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts src/services/legacy-execution-recovery.test.ts src/services/recovery src/services/remote-workspace-save-concurrency.test.ts src/services/native-runtime/native-workspace-finalizer-failure.test.ts src/__tests__/native-workspace-finalization-work.test.ts` passes (563 tests). - The transport tests use real sandbox commands. They prove an occupied folder reports `stage=inspect_root reason=occupied_root` and keeps its files. They prove a wrong manifest reports `stage=extract archive=manifest_mismatch`. They prove a missing branch reports `stage=checkout_branch`, and a changed origin reports `stage=verify_origin`. Forged output with a path is dropped. - The real-PostgreSQL cold-restore test proves an occupied root gives a non-retryable `sandbox_restore` failure and keeps the unknown files. - The completion tests cover retried, exhausted, refused, post-dispatch, and `in_progress` cases. - `tsc --noEmit` for the server reports no errors in the changed files. ## Risks - A retry keeps the same restore path. If the cause is deterministic but unnamed, the run uses its two retries before the board action. Named refusals skip the retries. - The board action is recorded only for statuses that stranded-work recovery does not scan, and only when no other action is active. This prevents a second recovery identity for one failure. - Script stderr grammar is new. Old sandboxes run the script that the server sends, so no runner update is necessary. - No migration, deployment, or credential change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use, code execution, and browser tools for read-only staging evidence. Context window and reasoning mode are host-managed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
833d068741
commit
beb73381ca
6 files changed
+322
-13
No files matched your search
@@ -2154,6 +2154,23 @@ is the current review participant. Before the receipt, the hold prevents a secon
|
||||
executor on the workspace. A snapshot-only save does not release the hold,
|
||||
because the task workspace does not contain the saved files.
|
||||
|
||||
A Stop or reassignment can reach a native run while its provider session is
|
||||
still opening. The startup owner then fences the late handle before any prompt,
|
||||
and the Stop acknowledgement records `nativeCancellation.beforeProviderTurn`.
|
||||
A reattached provider from restart recovery never gets this proof. When the run
|
||||
also has no runner event outside the startup lifecycle (harness, session, workspace,
|
||||
runtime phase, credential refresh), no turn or provider event, and
|
||||
no native result, no provider outcome
|
||||
is unknown. The completed copyback receipt then releases the hold for the stopped
|
||||
run's own agent as well, so the user's redirect continues once. The hold row and
|
||||
its evidence are unchanged.
|
||||
|
||||
A queued turn for the same task that never started is not a workspace writer for
|
||||
automatic recovery. The hold gates its dispatch until the recovery receipt is
|
||||
complete, so recovery and that turn cannot both wait for each other. Running
|
||||
turns, and queued turns for other tasks that share the workspace, still block
|
||||
recovery.
|
||||
|
||||
The deadline is a backstop, not the normal failure path. When native task
|
||||
preparation fails after a fresh controller claim and before any provider turn,
|
||||
the controller stops renewal and releases only that exact claim. Ordinary
|
||||
|
||||
Reference in new issue
Block a user