mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 15:55:46 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The workspace recovery worker saves the files of failed runs. One passive pass of the worker visits holds with obligations: saved input that waits, cleanup, and due re-arms and fallbacks of exhausted holds > - #17741 restarts that pass after each committed workspace-recovery change, so new work is found quickly > - On staging, changes commit continuously (stop receipts, new holds, recovery progress). Every pass restarted from the first candidates, which are settled holds whose saved input waits behind another hold, and it never reached later candidates > - After #20295, #20569 and #20661 were deployed, no due re-arm ran: five stop-proof holds and about twenty capture-policy holds kept their exhausted state, with no marker and no activity > - This pull request restarts the pass only after it reaches its tail, and it continues an unfinished pass on the next run > - The benefit is that every candidate is visited on a busy instance, so the exhausted holds re-arm, save their files and free their tasks without an administrator ## Linked Issues or Issue Description Refs #17741 Refs #20295 Refs #20569 Refs #20661 **What happened?** Exhausted workspace holds were not re-armed on staging, even after the fixes that add their re-arm were deployed. Their recovery actions stayed unchanged for days (`updatedAt` on Nov 5 or Nov 10). No `stopProofRecovery` or `capturePolicyRecovery` marker and no `issue.workspace_recovery_rearmed` activity was recorded. The worker did process new restore holds, which are selected by its unfinished pass. **What did you expect to happen?** The worker's passive pass reaches every hold with a due re-arm or fallback, re-arms it, and saves the files. **Steps to reproduce** 1. Create several settled holds with saved user input that waits, with action ids that sort before an exhausted stop-proof hold. 2. Start the real workspace recovery worker. 3. Commit a workspace-recovery notification every 50 ms, as stop receipts and new holds do on a busy instance. 4. The worker never re-arms the exhausted hold. Without the notifications, it re-arms the hold. ## What Changed - `workspace-recovery-work.ts`: a committed change marks the passive pass for restart (`restart`). It no longer resets the cursor. While a restart is pending, an unfinished pass that made progress continues on the next run. - `workspace-recovery.ts`: when the passive cursor passes the last candidate and a restart is pending, the same run starts a fresh pass from the first candidate, so a new change is not delayed by another run interval. Without a pending restart the pass records `complete` and does not wrap, so paused input does not keep the worker active. - `doc/execution-semantics.md`: document that a pass is never restarted midway. - `workspace-recovery-stop-proof.test.ts`: add a real-worker regression in the staging shape (six earlier settled holds with waiting user input, one exhausted stop-proof hold, continuous notifications). ## Verification - Red: on master, the new test "re-arms the hold on a busy instance whose passive sweep keeps selecting earlier settled holds" fails (`stopProofRecovery` stays undefined). Control: the same test without the notifications passes on master. This shows that the restarts cause the starvation. - Green: with the fix, the test passes. These suites pass: `workspace-recovery-stop-proof` (12), `execution-control-work` (27), which includes "visits a newly committed earlier Retry during a busy recovered-source pass, then becomes quiet", plus `workspace-recovery-capture-policy`, `workspace-recovery-preparation-dispatch`, `workspace-recovery-warm-handoff`, `workspace-recovery-preparation` and `server-startup-feedback-export` (279). The timing-sensitive `heartbeat-observed-workspace-preservation` suite passed 6 of 6 runs (an earlier revision that restarted only on the next run made it flaky). - Staging evidence (read-only GET requests): the 28 exhausted holds of the company have no re-arm marker and no fallback check. Many blocked tasks with settled holds have deferred wakes. ## Risks - A change committed during a pass is now picked up at the next pass, not at once. Unfinished physical work still uses its own pass first, so new recoveries are not delayed. Saved input in a passive obligation can wait up to one full pass. - Previous-release story: no persisted state changes. Holds saved by earlier releases are selected on the first full pass after deploy, with no migration. - Quiet behavior stays: a completed pass with no new change keeps the worker idle. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, code editing, browser GET reads) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>