mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 22:56:58 +01:00
97188f276c70a4cb033cb2a93f2b7339f796b1ab
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
bee0f0e2fa |
fix(server): recognize recoverable result finalization during upgrades (#19651)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native execution saves an accepted result before it finishes
workspace export and status projection.
> - The upgrade report currently recognizes only a live execution in the
observed phase.
> - A durable accepted result can therefore block an upgrade after its
execution lease is released.
> - Existing finalization owners already resume that saved work with
task binding, physical worker stop proof, and an atomic owner claim.
> - This pull request lets the report inspect those existing contracts
without taking authority.
> - Qualified remote work can continue through a normal server
replacement.
## Linked Issues or Issue Description
Refs #18037 and #19596.
**What happened?**
An accepted remote native result remains in result acceptance, workspace
finalization, retry, or committed projection. The upgrade report marks
it blocking because it is no longer a result-less observed execution
with a live execution lease.
**Expected behavior**
Report a saved result as recoverable only when the existing replacement
owners can resume it. Preserve all task, result, contract, remote
workspace, provider lease, physical owner, decision, and workspace
barrier checks. Keep unsupported or unknown work blocking.
**Steps to reproduce**
Run the new PostgreSQL post-result readiness fixture. It persists a real
encrypted Runner header and an accepted terminal result with a bound
remote workspace. The unchanged report returns one blocker.
**Paperclip version or commit**
Regression base:
|
||
|
DottaandPaperclip
|
c09fb60ca6 |
fix(recovery): retain worker-stop audit in preparation retries (#19612)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Recovery must preserve saved files before it admits another execution. > - A failed preparation can need a verified stop of its previous workspace worker. > - Recovery keeps that verified stop receipt in the run profile for audit. > - The preparation retry guard treats the retained receipt as unknown execution evidence. > - This pull request recognizes a valid same-run audit receipt while it retains all current execution and file checks. > - A saved Retry can then continue once after recovery completes. ## Linked Issues or Issue Description **What happened?** A native preparation failed before provider dispatch. The recovery worker verified that the previous workspace worker had stopped and saved its files. The same HTTP Retry stayed deferred because the run profile retained `nativeWorkspaceOwnerStopProof`. **Expected behavior** The saved Retry should continue once after all preparation, stop, file, ownership, and authorization checks pass. The old run and its audit receipt should remain unchanged. **Steps to reproduce** 1. Fail native preparation with a retained remote workspace and no provider activity. 2. Let the recovery worker acquire the old workspace owner through verified platform stop evidence. 3. Submit one HTTP Retry and let preservation complete. 4. Observe that the old guard retains the retry hold even after saved-file verification passes. **Paperclip version or commit** Reproduced against `d728be8fbf`. Related work: #18908, #19021, #19026, and #19466. This change covers the retained workspace-worker stop audit left by a later repair. ## What Changed - Add a typed reader for a host-persisted same-run worker-stop audit receipt. Keep signature and expiry validation at ownership acquisition. - Recognize only that validated audit field in failed-preparation settlement. Keep unknown profiles and active owners blocked. - Exercise real preservation, HTTP Retry delivery, native dispatch, saved files, accepted results, and task completion. Add malformed, foreign-run, unstopped-worker, and active-owner cases. - Document the distinction between retained audit evidence and current execution authority. ## Verification - Failing before the fix: the current-worker preservation case passed; the stopped-worker case produced zero successor runs after the same Retry, where one was expected. - Passing after the fix: 47 recovery cases, 10 native dispatch/preservation cases, and 6 platform-proof tests. The two preservation success cases overlap between the first two groups. - Server typecheck passed all seven layers after the final test changes. The Runner and SDK dependencies were built in this isolated worktree. - Final comment-free commit `44f8f33f8d917dddb51852283a479786102eab4d`: a fresh combined run passed all 61 affected cases in 67.92s. Fresh server typecheck passed all seven layers. - The earlier `95d0376c` combined run passed 60 cases and hit one `socket hang up` in the existing `catalog_provider_warning` HTTP test. That case passed in a bounded isolated run (12.48s), and the final 61-case combined run passed. The socket failure remains recorded as an intermittent HTTP harness limitation. No retry or weaker assertion was added. - Final commit `44f8f33f8d917dddb51852283a479786102eab4d` completed all required CI gates: 57 checks succeeded, 3 intentional skips, and the advisory Paperclip review is neutral with a substantive 5/5 verdict. Greptile is 5/5 and there are no unresolved review threads. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/42293625696). The follow-up commit only removes six new code-comment lines to follow CONTRIBUTING. - The native provider and signed platform API are controlled test boundaries. The tests use real PostgreSQL, recovery, ownership, file persistence, retry admission, dispatch, and finalization services. Deployed recovery remains unverified until normal rollout. ## Risks - The main risk is mistaking an audit receipt for current stop authority. The reader checks its exact shape and run binding. The receipt cannot replace stopped leases, saved-file receipts, settled preparation, current authorization, or the absence of an active owner or provider evidence. - The negative matrix runs with the audit field present. Provider activity, unknown usage, bad file receipts, Stop, actor changes, and unsettled leases still block continuation. - No schema or migration change is required. ## Model Used OpenAI GPT-6 in Codex. The exact deployment model ID and context-window size are not exposed in this session. Used reasoning, code editing, shell execution, and test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a019d00a39 |
fix(server): wake native workspace finalization from committed work (#17734)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs can accept a result before workspace copy-back finishes. > - The periodic orphan pass repeatedly tries this recovery, including cases that require operator action. > - Copy-back needs exact ownership evidence. A generic timer cannot grant that authority. > - This pull request wakes workspace recovery from committed changes and local owner completion. > - Saved retries use the existing scheduler. Empty work, completed barriers, and operator holds become quiet. ## Linked Issues or Issue Description Refs #17717. This is slice 52 of the recurring-server sleep stack, targeting master after parent #17717 merged at `4c56702e06c698cc10d4a86514669aba0f4d9a9b`. Related work: #17709 (status reassessment), #17687 (unsafe export recovery), and #16478 (workspace repair holds). This PR covers native workspace copy-back and validation. Result commit/projection and resource settlement remain separate recovery duties. It does not claim full-instance sleep. ## What Changed - Register a company/run/task-scoped workspace worker. Reconstruct accepted, unfinished work from existing records. Use saved retry and lease deadlines. - Notify from both accepted-result writers (control-plane and PRP coordinator), workspace receipts, coordinator changes, operator confirmation, retained workspace repair, and relevant task updates. Notifications follow the outer transaction. - Distinguish active local ownership, busy external ownership, pending Cloud evidence, operator-required evidence, and changed bindings. Only concrete external ownership/evidence waits receive a scoped retry. - Preserve advisory locks, signed stop proof, exact binding checks, physical callback joins, and existing failure policy. Join physical operations during shutdown. - Respect the existing heartbeat execution-suppression policy before discovery and execution; wake suppressed work on settings activation. - Skip physical workspace recovery in the enabled periodic orphan pass. Keep direct/startup reconciliation and the remaining result/cleanup recovery paths. - Keep routine company-only finalization hints quiet. Ordinary work requires issue/run scope; explicit settings and owner-release events keep their company-wide contract. - Preserve the parent's entire Runner/package tree, profile 80 and all historical identities. Add the dated obligation, producer, lifecycle and limitation audit. ## Verification - Current local source: `4059b24e3ea5c623be54ade18ec9b6ab78eb0289`, prepared by a normal merge of the current parent. The sole manual conflict keeps ordinary workspace fanout inside the existing issue/run guard. All 1,198 parent-only, 15 owned-only and 10 automatic shared paths are preserved. The complete package tree matches the parent byte-for-byte. - Controlled regression: the original workspace-only unscoped fanout performs three candidate SELECTs for routine company hints. The same zero-query assertion passes with scoped fanout. A real accepted result then produces one successful workspace settlement and returns to quiet. The failed baseline remains in the audit. - The first complete local group passes **1,693 cases in 14 files**, including all 19 workspace-worker cases. It covers both accepted-result producers, physical ownership, export retry/failure, finalization recovery, arbiter corpus, issue recovery actions, control-plane port, workspace recovery, heartbeat recovery, startup, retained cleanup and native execution. - Separate process-recovery and external-chat response files qualify the changed recovery/settlement paths without sharing their general test database with other files. Both pass in full: 410 process-recovery cases in 162.79 seconds and 273 external-chat cases in 53.79 seconds. The total is **2,376 distinct passes in 16 complete files**, with no skips. Normal merge `5c46c52873158529736b8349e4d44658865108ee` adopts the parent squash with this entire tested child tree unchanged. - Production server typechecking, build, module boundaries, migration order and source-profile checks pass. Thirteen dedicated PostgreSQL 17 fixtures use the unchanged migration 413 schema. Embedded-only paths use the existing recovery helper with fresh per-call fixtures. All temporary imports are restored byte-exact. No test assertion or timeout is relaxed. - Runner validation belongs to the unchanged parent package tree: 130 passing installation/compatibility/transport cases, 22 installed ACPX/Grok contract cases and 26 native-provider Rust cases. One existing opt-in closed Pi snapshot case is skipped because its distribution root is not configured. These are source-scoped checks, with no live provider qualification claim. - The owner audit at `doc/plans/2026-10-26-native-workspace-finalization-sleep-audit.md` preserves earlier failures and their disposition. Current-head acceptance: all 57 checks are clean, including protected CI and security. Greptile gives 5/5 on `7df5c9c44ea9e171be3c61b4430d3727b10f3337` with no actionable findings. Earlier scores do not substitute for this result. No Ponytail verdict is present under the authorized missing-review policy. - No claim of a full local repository suite, deployed app/database sleep, expanded Cloud wake coverage or measured cost savings is made. ## Risks - Supported wake producers are in-process application services. This adds no cross-replica or independent SQL-writer transport. - A real unresolved export still needs recovery. Busy owners and pending Cloud stop evidence retain a 30-second scoped check. No timeout authorizes copy-back. - Result projection and resource release still use their existing recovery. Some existing cleanup failures remain best effort; this PR does not infer cleanup success from a workspace barrier. - Cloud alarm coverage and host-sleep safety remain unchanged. No deployed suspension or cost savings have been measured. - Parent #17717 is merged. Current base is master at `4c56702e06c698cc10d4a86514669aba0f4d9a9b`. Fresh current-head CI and review must pass before this slice merges. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. No subagent was used for this slice. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Codex <noreply@openai.com> |
||
|
|
0b0e53029f |
fix: recover saved workspace results automatically (#15824)
## Thinking Path - Paperclip preserves accepted agent results before workspace copyback and result commitment. - A replaced controller can leave a physical copyback owner receipt behind. A lost database lock or expired lease cannot prove that its tar/Git children stopped. - Infrastructure recovery should belong to the runtime and deployment harness; task users should not investigate processes or attest shutdown. - New receipts record the exact deployment. The harness independently verifies that deployment fully stopped and signs a short-lived proof bound to the stack, run, owner and deployment. - The existing guarded reconciler then finishes the same accepted result without a new agent turn. Legacy receipts retain the documented administrator API after independent platform verification. ## Linked Issues or Issue Description **What happened?** An accepted native result can remain at `workspace_finalizing` with an abandoned physical copyback owner. Repeated recovery sweeps could replace specific instructions with advice to start new agent work. The first version of this PR exposed a shutdown-evidence form in the task; that asks end users to resolve infrastructure details. **Expected behavior** Automatically verify and recover the exact stopped deployment, preserve the original accepted result and existing approval/monitor waits, and show only a quiet task status. Unknown or live workers must remain fenced. Older receipts without platform identity need independently verified operator assistance through the administrator API. **Steps to reproduce** 1. Persist an accepted native result and a copyback receipt from a prior controller. 2. Replace that deployment before workspace finalization commits. 3. Let the normal finalization reconciler revisit the saved result. 4. With a platform-bound receipt and a fully stopped deployment, recovery should finish automatically. With missing, live, unverifiable or changed ownership, it must not write files or create another provider turn. Related public work: #14402 introduced copyback ownership; #9687 handles dead-run finalize barriers; #15795 handles terminal stale-run cleanup. This change addresses an accepted result whose physical copyback owner prevents commitment. ## What Changed - Record Railway project, environment, service and deployment identity on new physical copyback owners. - Request read-only platform verification from the managed cloud harness. Require an Ed25519 stop receipt with a disjoint purpose, configured signing key, exact stack/run/owner/deployment binding and valid expiry. - Keep live, unknown, malformed, unavailable and legacy owners fenced. Compare the whole prior receipt during ownership takeover, under the existing dedicated advisory lock. - Persist platform proof and audit automatic takeover. Resume normal finalization of the same saved result, preserving task ownership, approved boundaries, monitors and pending confirmations. - Remove process metadata, shutdown evidence, confirmation checkboxes and infrastructure-repair forms from task views. Suppress the duplicate technical owner-recovery card. Show “Restoring saved work…” only from server-confirmed recovery or queued operator recovery; platform identity alone shows the operator-assistance fallback. Remove the status when recovery resolves. - Share current-task, current-running-execution, accepted-result, authoritative-reviewer and newer-execution checks between automatic and operator recovery. Validate after the provider lookup, atomically with the owner claim, and again before file publication; closed or superseded tasks cannot start copyback. - Keep the separately authorized exact-owner API for operator recovery of legacy receipts, with active-lock, live-controller, stale-result and newer-execution rejection. Generic dismissal cannot bypass the copyback fence. - Replace obsolete form stories with production-backed automatic, queued, legacy, completed and mobile examples; retain stable suite IDs. Update the operator runbook and scoped Storybook catalog. ## Verification - Red/green: new platform-proof tests failed before implementation; the database takeover regression failed against the original ownership guard; reduced-UI assertions failed against the original form. All pass after the changes. - Passed: 59 physical-owner/finalization database tests using isolated PostgreSQL, including real reconciler commitment from all three saved-result phases through both automatic and operator recovery. They preserve result identity, exact saved file bytes, pending approval and overdue monitor waits, and create no new run or wake request. Unknown evidence and changed ownership remain fenced. - Passed: 162 focused proof, cloud identity, recovery route, recovery-card and reduced-UI tests. - Passed: full workspace typecheck and build, final server/UI typechecks, token gates, and Storybook build. - Browser inspection: desktop task composition shows only the quiet status; completed example removes it and retains the approval-wait message. Mobile shows the existing bottom navigation with no horizontal overflow. These are fixture UI observations, not live provider acceptance. - A companion harness change implements the actual provider check, signed receipts, server-token authentication and bounded/coalesced provider reads. Its full typecheck and test suite passed: 3,240 passed, 92 skipped, zero failures. This application alone cannot enable platform verification. - The updated full application `pnpm test:run` was stopped after repeated local embedded-Postgres startup failures; it is not claimed green. The same affected recovery suites pass against isolated PostgreSQL. CI passed on final head `9ad5c3529e`: 54 successful checks, two intentional skips, and successful legacy Snyk status. Greptile is 5/5 with no unresolved findings. - Global Storybook catalog validation is green: 225 files, 2,224 stories. Its 17 catalog/index tests pass. Master was merged and the inherited index metadata was reconciled after source review. - Review correction red/green: two UI and two database assertions reproduced misleading progress / missing server recovery state; all pass after server-confirmed state was added. All 153 focused server tests and 68 UI tests pass on the final recovery code, including repeated-sweep idempotence. Full workspace typecheck/build, token gates and Storybook build also pass. - Safety regression red/green: stale-task copyback and publication tests failed against the previous guard. The final suite covers changes during proof lookup, closed/cancelled tasks, replaced execution/checkout, newer and queued legacy runs, reassignment and rejected/unbound results, plus the real reconciler selecting closed or superseded tasks. Files remain untouched and no workspace operation starts. - No staging mutations, deployment, live provider probes or live production recovery were performed. The three historical receipts lack deployment identity and still require independent operator proof. ## Risks Automatic takeover requires the companion harness release and configured runtime-identity keys. Deploying only this application leaves unavailable evidence fenced. Legacy receipts are not retrospectively assigned a deployment identity. An authorized operator can still make an incorrect attestation through the fallback API; its administrator gate and exact-owner safeguards remain. Provider history that is missing or deeper than the bounded lookup defers automatic recovery. Copyback can still fail under its existing retained-workspace policy. No migration, telemetry change or cloud provider credential delivery is required. The companion harness PR also has passing CI and a 5/5 review; neither branch is deployed. ## Model Used OpenAI Codex, GPT-6, with repository inspection, browser verification, tool use and code execution. The exact serving model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the relevant tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing> |