mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 11:59:26 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a run finishes but copying its files out of the sandbox fails, Paperclip holds the task and tries to recover the saved files. When recovery gives up, the task offers Retry. > - Some saved files can never be recovered: the sandbox that held them was destroyed, or Paperclip cannot prove which runs used a shared sandbox. Each Retry ran the same check again, failed again, and parked the Retry in a wait that never ends. > - The task told the person to press Retry again or to wait for a Paperclip update. That breaks the rule that recovery must work and that a Retry that is shown must work. > - This pull request makes Retry continue the task in a fresh session and a new workspace from the work that is recorded, when the saved files are provably unrecoverable or cannot be proven safe. The task says clearly that the unsaved files were not recovered. > - The benefit is that an offered Retry starts a run, and nobody waits for a fix that another click cannot bring. ## Linked Issues or Issue Description Refs #21315, #21295, #21267 **What happened?** On hosted staging, two blocked tasks offered Retry (`canRetry: true`). Each Retry was accepted, and within about 3 seconds it was parked in a wait. No run started. Every later Retry did the same. - Task A: the hold cause was `native_workspace_sync_out_retry_exhausted`. Recovery ended with `source_unavailable` (`use_alternative_strategy`). The sandbox that held the files had a signed absence proof and a `destroyed` termination receipt. Retry was parked with `saved_work_recovery_failed`: "Saving the agent's files failed. Retry to try again". - Task B: the same hold cause. Recovery ended with `allocation_ownership_ambiguous` (`service_required`), because the sandbox was a reusable sandbox that other runs also used. Each Retry opened a new recovery generation, ran the same check, and was parked with `saved_work_recovery_refused`: "Retry runs the same check again and succeeds only after Paperclip is updated." - In both tasks, the agent's turn had finished and its result was accepted. Its work was recorded outside the sandbox: a plan document and screenshot attachments for task A, and a pushed branch with an open pull request and an implementation document for task B. Root cause: admission had a branch for an accepted result whose files were not yet saved. When the recovery budget was spent, the branch always returned a `saved_work_recovery_*` wait. It did not check whether the files could ever be recovered, so the fresh-workspace path that admission already has was never reached. The automatic retained-source fallback did not help either, because it handles only the remote workspace v1 reference, and these runs use the native workspace sync v3 reference. Also, Retry opened a new recovery generation for an ownership refusal, which only repeated the same check. **Expected behavior** When the saved files are provably unrecoverable or cannot be proven safe, Retry continues the task in a fresh session and a new workspace from the recorded work. The old files are never applied and the source sandbox is not changed. No provider turn is replayed. The task says that the unsaved files could not be recovered. The Retry offer and admission agree. **Steps to reproduce** 1. Let a native run finish with an accepted result, and let its workspace copy-back fail (`native_workspace_sync_out_retry_exhausted`). 2. Let saved-work recovery end with `source_unavailable` (sandbox destroyed) or with `allocation_ownership_ambiguous` (shared reusable sandbox). 3. Press Retry. 4. Observe that the Retry receipt waits with `saved_work_recovery_failed` or `saved_work_recovery_refused`, and no run starts. Press Retry again and observe the same result. **Paperclip version or commit** Staging on `origin/master` after #21315 (`bbafa78fe2`). **Deployment mode** Hosted cloud (staging), remote Daytona sandbox runner. **Agent adapter(s) involved** `paperclip_runner` (native runtime runs). ## What Changed - `savedFilesUnrecoverable` and `unrecoverableSavedFilesFailure` (`system-recovery-state.ts`) are the one rule for saved files that no recovery can apply. The rule is true for a spent recovery whose failure is a permanent source refusal: `source_unavailable`, `allocation_ownership_ambiguous` or `historical_source_missing`. It is false for `source_binding_invalid`, because a new recovery generation can still verify that source. It is false for `archive_unsafe` and `snapshot_verification_failed`, because a fresh attempt still imports earlier saved snapshots and would fail before the agent starts. It is also false after a fresh attempt is recorded. - Admission (`explicit-native-continuation.ts`) no longer parks a Retry of an accepted result in `saved_work_recovery_*` when every live recovery of the run is unrecoverable. The Retry takes the existing fresh-workspace path (`fresh_isolated_workspace`). That path keeps all stop proofs, leaves the source sandbox and workspace untouched, and starts a fresh session. - When such a Retry starts, the task gets one system comment. The comment says that Paperclip could not recover the unsaved files, why, and that the task continues from its recorded work (comments, documents, attachments and pushed commits). The settled hold records the same text. - Retry (`task-retry.ts`) binds to an unrecoverable recovery generation. It does not open a new generation that repeats the same check. - The blocker text for unrecoverable files says that Retry continues the task in a new workspace from its recorded work. It no longer says that Retry succeeds only after an update. - `canAdoptNativeSourceRetention` and the permanent source-refusal lists moved into `system-recovery-state.ts`, so that admission, Retry and messages share them. The old modules re-export them. ## Verification - Red/green: `cd server && npx vitest run src/services/unrecoverable-saved-work-retry.test.ts`. The tests fail on `origin/master` sources (or on the earlier head for the snapshot case) and pass with this change. They are built from the two staging shapes (destroyed sandbox with `source_unavailable`, and a shared reusable sandbox with `allocation_ownership_ambiguous`): - offers a Retry that continues the task (blocker and task view agree, and the text names the fresh workspace) - starts a fresh run from the recorded work: the Retry receipt has no wait and a run, the strategy is `fresh_isolated_workspace`, the hold is settled with the fresh attempt, one system comment explains the lost files with no raw IDs, the source lease is unchanged, and no new recovery generation opens - keeps a Retry out of a fresh attempt that would import a succeeded snapshot whose archive is unsafe - `workspace-recovery-warm-handoff.test.ts` › "wakes physical preservation from HTTP Retry before passive history without replaying the provider" failed on an earlier head of this PR, which also treated `source_binding_invalid` as unrecoverable. It passes unchanged now. - Updated tests that pinned the old repeat-the-check behavior: `workspace-recovery-preparation-gates`, `task-retry`, `system-recovery-state`, `workspace-recovery-retry-copyback`. - Related suites pass: `system-recovery-state`, `workspace-recovery-preparation-gates`, `workspace-recovery-retry-copyback`, `workspace-recovery-warm-handoff`, `task-retry`, `saved-work-retry-eligibility`, `workspace-recovery-fallback`, `explicit-native-continuation`, `workspace-recovery-admission`, `task-interruption-query`, `native-workspace-sync-retention`. - `cd server && node scripts/typecheck-layers.mjs` passes (source and test layers). ## Risks - Medium-low. A Retry of an unrecoverable hold now starts a new agent turn in a new workspace instead of waiting. The turn starts only through the existing fresh-workspace admission, which keeps every stop and cleanup proof. - Unsaved files of the old run are not applied. That was already true; the change is that the task continues and says so. - `source_binding_invalid`, `snapshot_verification_failed` and `archive_unsafe` keep today's path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, browser for read-only staging API reads). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>