mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 23:43:16 +01:00
6ee17aae1c61886a77ae8570fc0d360689c4dc0d
2918
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
6ee17aae1c |
test: settle saved retry cleanup before idle assertion
Observe the specific succeeded retry before closing its task and draining all execution. Once execution has settled, advance bounded fake cleanup time while joining worker passes; retain the existing timeout, exact retry identity, output and zero-SQL assertions. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
822b173281 |
test: preserve retry producer boundaries in receipt recovery
Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix/sleep-forward-02b-security: fix: wake retry scheduling only when retry state changes |
||
|
Dotta
|
63c922a193 |
Merge commit 'c2ff3eb92113fb16713b270e34932636956ff16b' into fix/sleep-forward-02b-security
* commit 'c2ff3eb92113fb16713b270e34932636956ff16b': fix: wake retry scheduling only when retry state changes |
||
|
DottaandPaperclip
|
c2ff3eb921 |
fix: wake retry scheduling only when retry state changes
Separate retry-schedule notifications from ordinary queued-run eligibility. Preserve exact deadlines and prove native producer commit/rollback behavior without global retry scans on ordinary agent or issue edits. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
af7b7d3316 |
fix: park unchanged project budget execution waits
Lock and re-read project pause eligibility before publishing budget intent. Cover actual project Retry parking/release and concurrent pause admission without no-op hints. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
c0fe0feffb |
Merge commit '19a97bbe57d1ebd3691c70dd822488efe7e5d3e1' into codex/sleep-stack-c1
* commit '19a97bbe57d1ebd3691c70dd822488efe7e5d3e1': fix(server): park persisted deferred admission refusals |
||
|
DottaandPaperclip
|
19a97bbe57 |
fix(server): park persisted deferred admission refusals
Distinguish a budget or agent refusal saved on the existing receipt from genuine recovery failure. Preserve HTTP409 messages and details, grant each new wait reason its existing finite grace, and retain failure backoff for unsaved or unexpected errors. Prove actual Retry admission,25-minute SQL quiet, independent warm producer-to-process release, missing/cancelled receipts and real PostgreSQL write failure. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
290d12cf4b |
merge: rehearse issue-recovery predecessor for saved receipts
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dee5296b8a |
merge: rehearse current dispatch predecessor for issue recovery
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fcc107739c |
merge: rehearse current master and UI prerequisite for dispatch
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dff6c00468 |
Join receipt acceptance passes before advancing saved deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
0adba98c10 |
Merge commit 'e3fff84b6f81448c730dd165253a1a673f8d19b3' into codex/sleep-deferred-execution-receipts
* commit 'e3fff84b6f81448c730dd165253a1a673f8d19b3': fix: scope comment-triggered recovery to the affected issue test: await issue recovery dispatch and cancellation settlement |
||
|
DottaandPaperclip
|
c74f5f12ba |
Retain moved receipt deadlines and wake released execution gates
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e3fff84b6f |
fix: scope comment-triggered recovery to the affected issue
Preserve company reconstruction for graph, run and eligibility changes. A real PostgreSQL producer regression verifies one assignment check among1000 open tasks, with broader eligibility reconstruction still intact. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7bb1af657a |
test: await issue recovery dispatch and cancellation settlement
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ac6c5191ee |
Drive saved execution receipts from committed work and deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
87ae20ae65 |
fix(server): reconcile failed sandbox cleanup with signed absence (#18407)
Reconcile failed ephemeral Cloud leases through a signed cleanup-only absence observation when local historical inspection is unavailable. Preserve failure outcomes, exact resource bindings, retention and row-version fences; grant no run-continuation authority. All 54 current-head checks/statuses passed. Greptile is 5/5, Medium, with no findings or unresolved threads. The 38 focused tests pass; both new reconciliation cases fail without the fallback. Isolated baseline and branch idle suites each pass all 225 cases. The original broader local teardown timeout is retained in the PR, and the full local run remains in progress. Cloud protocol support is already serving in staging. Exact candidate composition and hosted provider sleep/wake qualification follow this merge. Production has not been changed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
0fe6e038c2 | Restore independent recovery duties and bound committed-source work | ||
|
Nicky Leach
|
9e904ee519 | perf(server): overlap independent heartbeat preparation reads (#18372) | ||
|
DottaandPaperclip
|
91261d5a03 |
fix(server): preserve budget history for deleted resources (#18389)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Budget views show saved policies, incidents and spend for each company. > - A policy can remain after its agent or project has been deleted. > - The overview currently stops when it cannot resolve that old resource. > - A current agent's budget then becomes unreadable during normal setup. > - This pull request preserves the historical row in display summaries. > - Admission and policy changes still require a real company-owned resource. ## Linked Issues or Issue Description **What happened?** The budget overview throws `Agent not found` when a saved policy references a deleted agent. One old resource prevents the company from reading its current budgets. The same problem affects historical project policies and incident summaries. **Expected behavior** The overview should retain historical policy and incident IDs and spend. It should display `Deleted agent` or `Deleted project` for a missing resource. Read access must not grant permission to execute work or change a budget for that resource. **Steps to reproduce** 1. Create an agent and a budget policy for it. 2. Permanently delete the agent after its other dependencies have been cleared. 3. Read the company's budget overview. The existing service throws instead of returning its remaining current and historical rows. **Paperclip version or commit** The isolated repair starts from merged commit `d9354bbfc35ef74333e90e596ce21dc8e59149b9` and integrates the exact merged baseline test fix `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. It also integrates the actual merged Antigravity auth fix `18b1637b6371a6a6adade857e2a73654ee626fef` (#18254). The diff against the selected merged parent remains two budget files. **Related work** Earlier reports and implementations: #11058 and #4333. This extracts the product prerequisite from #18351 so it can qualify independently of that journey's other dependencies. The change contains only the budget service repair and four database regression cases. ## What Changed - Resolve deleted agent and project scopes only when building display summaries. Preserve their original IDs, policies, incidents and accounting records. - Reject a saved policy or incident that resolves to another company's resource. Propagate other read failures. - Keep missing-company resolution, work admission and policy changes strict. Validate company, policy and incident scope bindings before either incident action. A budget raise requires a live company-owned resource. Historical dismissal may proceed only for a genuinely deleted owned agent/project, and must reject live foreign or mismatched scopes. - Add four canonical PostgreSQL regression cases: deleted agent history, deleted project history, missing-scope admission and budget raises with safe historical dismissal, and foreign/mismatched references. The history cases also verify current ledger spend and unchanged stored rows. ## Verification - Real-service RED/GREEN against official PGlite with every canonical migration: the original overview throws for deleted history; the repair returns the historical IDs, incidents and spend while retaining current budgets. - Mutation regression RED/GREEN: missing-resource budget raises must reject before writes; owned deleted-scope dismissal must reject its linked pending approval, retain policy/agent/ledger data and incident history, and remove the incident from the active list. Live foreign and mismatched scopes reject both actions without mutations. - Current integrated head `b3434f5ad05fd3067379352700c36bfe826179c4`: all four canonical PostgreSQL regressions pass in [server 2/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194077), and all five existing budget-service tests pass in [server 7/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194080). Both use the CI merge against selected parent `18b1637`. - Clean-build declaration typing passes in the current CI typecheck gate. The earlier local source-alias no-emit check also passed, including the canonical actor argument. The local worktree borrows compiled dependencies from a sibling checkout; a later real-service import was blocked by its stale `sandboxCreationOperations` export, so clean-build CI is authoritative for the final integration. No production workaround was made. - `git diff --check` passes. Broad repository typecheck, tests and build run in current-head CI rather than repeated locally. Native local PostgreSQL could not start because this host exhausted its semaphore pool; no global IPC settings or unrelated processes were changed. - One unchanged Runner lane retry was authorized after a probable intermittent close-settlement fixture failure. The [original failure](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124360234111) occurred in `omitted-context` cleanup. A [fresh passing Linux lane](https://github.com/paperclipai/paperclip/actions/runs/41142552195/job/124360110386) passed the same case and all 1,998 tests with byte-identical entire Runner and adapter-utils trees, lockfile, root package, scripts and workflows, against the same merged base. The fixture has a five-second close allowance with a 2.5-second suspension reserve; a host stall is not proven. No budget path or changed close input was found. The [single targeted retry](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124365812609) passed the same case in 1,034 ms and all 1,998 lane tests; dependent verification also passed. All 57 current PR checks are terminal and green. - The product helper requires all current CI and clean review before merge. Actual QA budget-overview proof will run on the combined final app before hiring qualification. No deployed success is claimed. The older `b4863dd` artifact was never deployed; selected parent `18b1637` adds sandbox/lifecycle changes and migration 0418, so whole-app identity is not claimed. The budget module and relevant budget contracts remain unchanged, and final hiring requires the combined final app with explicit source identities. ## Risks Budget summaries now return historical rows whose resources have been deleted. These rows use explicit display names and preserve their accounting data. They do not create a resource or relax work admission, authorization or budget grants. Historical incident dismissal changes only that incident and its linked approval; it does not resume work or raise a budget. Cross-company references still fail closed. There is no schema migration and no new background work. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and database test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the PR description documents the display-only history and strict-mutation contract) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
9eee45ed2d |
Merge branch 'codex/sleep-run-task-recovery' into codex/sleep-issue-state-recovery
* codex/sleep-run-task-recovery: fix: normalize run dispatch hints to company identity # Conflicts: # server/src/services/run-dispatch-notifications.ts |
||
|
DottaandPaperclip
|
972fbc0760 |
fix: normalize run dispatch hints to company identity
Wider interaction and execution objects must coalesce into one company dispatch scope. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
ab5108611d | Run recovery producer acceptance on isolated PostgreSQL fixtures | ||
|
Dotta
|
ff5586cb74 |
Merge branch 'codex/sleep-run-task-recovery' into codex/sleep-issue-state-recovery
* codex/sleep-run-task-recovery: fix(environments): persist sandbox ownership before creation (#18357) fix(server): bound failed lifecycle verification (#18383) |
||
|
Dotta
|
dc4456c76b | Drive issue recovery from scoped source writes and saved deadlines | ||
|
DottaandPaperclip
|
7db456e235 |
Merge origin/master and retain both Cloud skill fixture contracts
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
06c28d4d2f |
Merge branch 'codex/sleep-run-task-recovery' into codex/sleep-issue-state-recovery
* codex/sleep-run-task-recovery: test: prove timer deadlines survive settled drain holds fix: back off expired deadlines during domain drain perf(heartbeat): let the dispatch worker own startup retry promotion fix(server): scope run retention hints to affected companies fix(heartbeat): scope and batch dispatch wakeups by owner fix(evals): support official TestOps job metadata (#18385) fix: release settled deadline work holds during drain fix(agents): revalidate error-clear authority before write intent test(skills): verify Cloud CLI reads through shared object storage # Conflicts: # server/src/services/issues.ts # server/src/services/run-dispatch-notifications.ts |
||
|
DottaandPaperclip
|
228196a110 |
test: prove timer deadlines survive settled drain holds
Expect committed producers to release idle admission while retaining a resume deadline, then verify actual dispatch and final quiet after resume. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
45a3a92e6d |
fix(environments): persist sandbox ownership before creation (#18357)
## Thinking Path > - Paperclip manages companies and the agents that do their work. > - Remote execution creates provider resources that need a durable owner. > - The server could create a sandbox before it saved its lease. > - A failed lease write could leave cleanup data only on the old web server disk. > - This change saves a company-scoped operation before creation and transfers ownership atomically. > - Recovery finds the original Daytona resource or records an explicit manual reconciliation outcome. ## Linked Issues or Issue Description **What happened?** A sandbox could exist without a lease after a database failure or company rebind. Failed cleanup recording could then create a local fallback file. Replacing that web server could lose the cleanup reference. **Expected behavior** A failed admission write makes no provider call. An interrupted creation retains a committed operation. Recovery must not create a second sandbox or destroy an attached lease. **Steps to reproduce** 1. Run the disposable PostgreSQL cases in `server/src/__tests__/sandbox-creation-operations.test.ts`. 2. Reject the lease insert after the controlled provider creates its resource. 3. Start a new receiver process with a different empty web disk. It must find the same resource and create one cleanup owner. Related ownership report: Refs #14672. This change addresses new allocations before lease persistence. It does not replace existing lease recovery. ## What Changed - Add an idempotent migration and a small creation-operation table. Retain original company, environment and run identities across deletion. - Commit creation intent before run, ad-hoc lease, and plugin task replacement calls. Fence duplicate requests and ambiguous lease commits. - Retain a known returned task resource ID in the existing operation before attachment. Preserve manual reconciliation and never delete an ambiguously attached resource. - Keep company and run authority checks and atomic lease transfer with the lease owner. - Reuse Daytona names, labels, read-only inspection and claimed cleanup. Keep provider payload handling in the provider. - Retain nonsecret credential identity hashes. Refuse recovery lookup under a changed credential value. - Record manual reconciliation for missing/deleted or explicitly inactive saved credentials, versions, and provider vaults. Keep temporary resolution failures retryable. Never reactivate a disabled credential. - Use transaction signals and concrete deadlines. Failed records back off independently. Manual reconciliation stays durable and quiet. - Remove new orphan-file fallback writes from the covered paths. Keep legacy file replay and its tests for cutover. - Replace obsolete fallback-producer assertions with normal-entrypoint durability tests. Keep cleanup, diagnostics and warm handoff coverage. ## Verification - [Independent current-head verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534744320) passed 190/190 combined tests, 40/40 incoming TestOps tests, full workspace typecheck, and build. PR head `958af986130f25e136065ad30d1fdd843c46255c` has tree `e6c3e5b02182d5941a70ccf5bf55638df3c2e28d`, identical to tested combined commit `b9d7a9c2e9b30f0153685fb24a5f70b56132201f`. Real PostgreSQL, compiled Rust Runner and controlled object providers were used; external-S3 integration remains outside this local proof. - Current head `958af986130f25e136065ad30d1fdd843c46255c` normally merges master `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. The sole conflict was the maintenance test: a parameterized case now preserves both legacy-local `local_storage` and valid shared-S3 `provider_mismatch` checks, including the parent local receipt, exact version identity, missing-snapshot/no-seed refusal, and unchanged persisted row. The three master TestOps files remain intact. No production, schema, migration, or package changes were made after the fully qualified prior head `0b4d05505e`. Independent review approved the resolution; the full managed-skill-content file and Cloud guard controls pass 58/58 (105.11 seconds). [Required PR CI passed on this exact merge head, first attempt](https://github.com/paperclipai/paperclip/actions/runs/41136987412): 54 successful checks and three intended skips. The additional Paperclip review is neutral at 4/5; two external Superagent checks (Contributor trust and Security Scan) remain in progress with zero annotations and no assessment, so they are not passing scan evidence. Greptile reviewed exact head `958af986130f25e136065ad30d1fdd843c46255c` at 5/5; the final thread audit has zero unresolved threads. All completed evidence below retains its recorded revision scope. The additional Paperclip Review on `958af98613` is neutral at 4/5 with a Ship verdict and no must-fix or should-fix item. Its optional observation is that a generic create rejection remains pending/manual even when an operator believes allocation did not occur. This change preserves uncertainty because the generic create-error contract does not prove provider absence; no new error classification or provider-specific compensation was added. - [Independent final-source verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534328247) passed all 189 real-PostgreSQL/Runner and controlled-fixture cases, full workspace typecheck, and build. The integration commit `c8411a8528386f4824b340ec5fdaaed1efb9ea69` has the same source tree as final PR head `0b4d05505e` (tree `bda65efc14b4c904d0f589154db77bb160641c8c`). Controlled object-provider coverage passed; the separately guarded external-S3 suite was not run because the bounded local MinIO pull was unavailable. This includes the two final compatibility fixes. - The first final-head CI attempt exposed one E2E timing failure in `disconnect-write-recovery.spec.ts:181`: the title stayed unchanged through the 15-second session-recovery poll. The same original case passed in 13.8 seconds on immediately prior head `7ec06f77b7`, and all UI/session/E2E source is identical across those heads. The failure log and CI artifacts are retained; no cause is claimed. One failed-job-only retry is authorized on this evidence, without changing a timeout, assertion, or production behavior. [The single failed-job retry completed successfully](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2): all 35 E2E shard cases passed (6.6 minutes), including the original session-recovery case in 14.0 seconds. The initial failure remains disclosed and its cause remains unproven. - Final head `0b4d05505ee0aa72f2981cdff74761406960ec73` adds two independently reviewed integration fixes after the complete 7ec CI matrix exposed two deterministic failures. Server 9/12 failed legacy local-storage maintenance because the CLI reached the new Cloud server startup guard; server 5/12 found the Daytona package version did not match its bumped manifest. The failed run is retained at https://github.com/paperclipai/paperclip/actions/runs/41131204203. It is not passing evidence and was not blindly retried. - Maintenance now constructs the existing explicit storage dependency after loading the required persisted Cloud snapshot, and passes it to both maintenance owners. It can verify legacy local storage before provider cutover while normal Cloud server startup remains guarded. The original case reproduced red locally, then the complete managed-skill-content and managed-cloud-prerequisites files passed 57/57 (32.05 seconds), including missing-snapshot refusal, unchanged stored configuration, and existing server guard controls. Daytona package version now matches manifest 0.1.10; the complete bundled package identity file passed 14/14 (3.98 seconds), and explicit manifest/lifecycle controls passed 2/2 (328 intentionally filtered; 3.42 seconds). An initial workspace-filter command matched no project and is not counted as pass evidence. The final diff changes no sandbox runtime or schema behavior. [Full CI is green on this exact final head](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2): 56 successful checks, three intended skips, one additional incomplete/neutral Paperclip Review, and no pending or failed checks. The neutral review is not a passing assessment. [Greptile reviewed the exact final head at 5/5](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6529396969), with no new actionable issues and all review threads resolved. - Final combined head `7ec06f77b7c14a745c569d18401f16df3d26b626` includes master managed-skills migration 0417 and shared-storage prerequisites from `198218bf55` through normal merges. Sandbox creation moves to generated `0418_curved_lady_mastermind.sql`; its idempotent SQL is byte-for-byte unchanged. The regenerated 0418 snapshot contains both sandbox creation operations and the master skill mutation receipts, and points to the master 0417 snapshot. Master SQL and journal history remain intact. Exactly five snapshots remain, 0414 through 0418. The only sandbox-specific changes since the approved credential policy are migration metadata/number and the matching test/document references; the sandbox runtime files are unchanged. - [Independent final combined verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6533960026) passed on integration revision `e6f05b4cf0cd`, containing final sandbox head `7ec06f77b7` and actual master `198218bf55`: migration/declaration checks, all 158 real-PostgreSQL/Runner and controlled-fixture cases (251 seconds), full workspace typecheck (81 seconds), and build (35 seconds). - Final combined migration proof: fresh real PostgreSQL has both tables. A separate original-0417 sandbox table with a retained pending row survives two applications of the renamed 0418 SQL unchanged, with the expected primary/index and foreign-key structure. Migration numbering/safety checks pass. The normal runtime migration reapply case passes 1/1 (20 intentionally filtered cases; 46.69 seconds). The first temporary qualification harness stopped before any database assertion because its external `.ts` file selected CommonJS; the `.mts` harness corrected that setup and passed. Independent review approved the exact metadata and reference diff. Earlier full suite and live-provider results below retain their stated revision scope. - Credential-policy head `c606d473b03709f27577a55a1dfc0b0d3b4359fd` handles the remaining inactive-credential review warning. The existing boundary accepts only concrete secret-owner 422 codes and exact no-details disabled/coming-soon vault messages. Unknown 422, authorization errors, provider wrappers, and temporary database/provider failures still propagate. Independent source review approved this three-file correction. No generic resolver, schema, provider protocol, retry cap, or activation behavior changed. - Creation/cleanup qualification at `c606d473b0`: all 39 cases pass against real PostgreSQL (81.27 seconds). The new cases use actual disabled/archived saved secrets, a valid disabled version, and provider vaults created ready before normal API updates disable them or mark them coming soon. Each remains manual through restart/repeated scans, with zero provider lookup and no retry deadline. The independent plugin idle blocker remains intact. The temporary 503 control retains retry ownership and later recovers one cleanup owner. Before this correction, all five inactive-state cases remained pending; that failed evidence is retained. The prior 18-case known-ID task proof below belongs to `237144fcf6`, whose helper and task path are unchanged by this correction. - Known-ID head `237144fcf69a249c0e0310563428c0f55ed6d55c` also addresses the independent task-replacement reliability warning. The existing creation operation now saves only the known nonsecret returned resource ID before expiry validation and attachment. The write is scoped to operation/company/original environment and allows only a missing or matching ID. It changes no ownership state and copies no resolved configuration, credentials, or provider payload. Independent source review approved this four-file fix. - The complete real-PostgreSQL task suite passes 18/18 (9.52 seconds). Rejected attachments retain the ID through manual reconciliation and repeated scans. A real committed attachment followed by an injected lost response retains the same active lease and attached operation, with no release or second create. Same-ID retries work; conflicting IDs and foreign companies cannot change it. The initial three regression cases failed because the ID was absent, before the fix. - The shipped Daytona manifest does not declare task support, so it cannot enter this generic replacement path. The dispatcher requires both declared task support and an environmentTask worker method. Generic drivers without creation lookup still use manual reconciliation, with a known returned ID retained when the database write can commit. No automatic lookup capability or unconditional compensation was added. - Previous boundary-fix head `8ba77def151ea85eeae2db7693254db7f87673b0` fixes the fresh review finding about missing credentials. Only known secret-owner 404 results at the recovery secret-resolution boundary select the existing manual outcome. Provider inspection and transient database/provider errors remain retryable. Independent source review approved this change. - Secret-boundary focused acceptance at `8ba77def15`: all 34 cases pass against isolated real PostgreSQL. The complete creation suite passes 16/16 (100.75 seconds); the complete cleanup coordinator suite passes 18/18 (99.83 seconds). The new cases use normal saved-secret creation and deletion, confirm zero provider lookup, restart/repeated scans, no automatic deadline, preserved independent plugin idle blocking, and no additional idle blocker after the controlled worker supplies its quiet proof. A temporary 503 resolution failure preserves pending ownership and later recovers one cleanup owner. No additional live provider resource was created. - Failed attempts remain part of the evidence. The initial red run reproduced the pending-operation bug and also exposed a numeric-deadline test harness mistake, which was corrected. A later combined run reported an existing replacement-receiver failure and a new idle-fixture assertion failure; it was interrupted while the next test file remained in imports. An isolated diagnostic confirmed that the manual state was correct but the installed fixture plugin correctly blocked sleep without its independent quiet proof. The corrected test retains that blocker assertion before supplying the controlled worker witness. The unchanged replacement-receiver case passes in the final complete 16-case run; its earlier failure is not counted as a pass or assigned a proven cause. No timeout or production idle guard was relaxed. - Greptile reviewed previous combined head `7ec06f77b7c14a745c569d18401f16df3d26b626` at 5/5, with no unresolved review threads. Its CI produced the two concrete integration failures addressed above. The earlier CI and live evidence below remain historical evidence, not a claim that the current full gates have completed. - Master integration update: merged `dbff3747b8` and then latest master `76fde891cc` without rewriting branch history. Only `doc/DATABASE.md` conflicted; the sandbox ownership, object-backed run evidence, and cloud configuration sections all remain. The master merges changed no sandbox implementation, migration, or provider code. Migration numbering/safety checks pass and exactly five schema snapshots remain. At merge head `f5d23d4a59828835d01c150f83fa2645fb2d3bec`, the two focused creation/coordinator suites pass all 32 cases against isolated real PostgreSQL (64.57 seconds). Those merge-head checks precede the missing-credential fix below. The earlier live Daytona proof still covers the unchanged normal creation/lookup/teardown path; the new missing-credential branch is qualified with real PostgreSQL and controlled provider hooks. The optional Superagent Security Scan was inconclusive on prior head `7ec06f77b7`: its pinned PR merge-base checkout failed or timed out, with a neutral conclusion and zero annotations. An earlier scan could not start within 120 seconds; a bounded rerun request returned HTTP 404. These scanner failures are not passing security-scan evidence. On final head `0b4d05505e`, the additional Paperclip Review check is neutral: its assigned review execution did not complete and produced no validated assessment. Fresh Greptile 5/5 and independent source review are separate evidence. Previous-head required CI, independent review, and the live Daytona qualification below passed. - [Complete previous-head CI](https://github.com/paperclipai/paperclip/actions/runs/41105726052) passed at `0b5cbb763ab8faa25aa0ff1d8a405cdc0d3db409`: 53 successful checks and three intended skips, including typecheck, full tests, build and E2E. Greptile is 5/5 on this head and both original findings are resolved. - Initial head: 200 PostgreSQL runtime, creation and coordinator tests passed. The corrected head also passes all 32 creation and coordinator tests, including deletion before/during lookup, a failed lookup after deletion, restart, settled polling and actual idle-safety checks. Cases cover failed admission, failed lease writes, concurrent recovery, lost provider replies, ambiguous transfer retries, company rebind, environment deletion, stopped runs, lookup backoff, restart, drain and settled SQL quiet. - Separate receiver processes: passed. A controlled HTTP provider retains the resource while three independent receiver processes use two empty web disks. Recovery creates one cleanup owner and never repeats creation. - Fresh PostgreSQL migration and two additional applications: passed. The pending creation row survives unchanged. - Server TypeScript and SDK/shared build checks pass after the corrections. Migration numbering/safety checks passed. Exactly five schema snapshots remain. - Daytona adapter: all 330 tests passed, including stable-name lookup, ownership checks, deletion and the updated manifest version. An earlier run exposed the version expectation and it was corrected. Previous-head CI passed. - Live Daytona acceptance passed on `0b5cbb763a`: isolated real PostgreSQL, normal runtime acquisition through independent receiver processes, actual Daytona plugin/SDK 0.203.0, one disposable Ubuntu 24.04 sandbox (1 CPU / 1 GiB RAM / 1 GiB disk). A forced lease insert failure retained the committed operation; a fresh receiver with empty disk found the same sandbox, repeated recovery retained one cleanup owner, and the actual claimed cleanup sweep expired the lease and resolved the operation. Independent SDK lookup returned typed not-found; ordinary recorded-ID inspection returned destroyed. This successful attempt made one create, one recovery lookup, and one teardown. Two earlier failed harness/provider-confirmation attempts are disclosed in the [live acceptance record](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6530313054). The coordinator independently queried all three exact resource IDs through the SDK and confirmed typed not-found for each. - Complete CI-affected heartbeat/plugin warm-task suites passed 21/21 after test corrections. An unchanged workspace exposure case passed in isolation after the prior CI shard reported an unhandled readiness rejection. Complete previous-head CI and refreshed review passed. ## Risks - Deleted companies retain an explicit manual reconciliation outcome. Recovery does not retry missing credentials or insert a lease for a deleted company. - Only Daytona declares automatic creation lookup. Other uncertain providers retain `needs_reconciliation`. A missing creation name alone cannot prove that a late allocation will never appear. - Pending operations use the existing process-local signal and startup recovery model. This does not add cross-replica wake delivery or prove deployed sleep. - Legacy direct provider probes and custom-image setup use separate lifecycles. They are outside this replacement of the run-lease orphan-file fallback. The normal Test path that acquires a runtime lease is covered. - Existing orphan files still require the separate cutover inventory and settlement. This change does not delete them. - The schema change is additive. The new SQL can be reapplied. Historical SQL and journal entries are preserved; only the oldest generated snapshot is pruned. ## Model Used OpenAI `gpt-6.1-sol`, with tool use, code execution and repository review. The harness did not report the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge The required branch status checks (`ci / verify`, `ci / e2e`) and all test/build jobs passed on this head. The non-required Superagent Contributor trust and Security Scan checks are still running without findings; they are not claimed as passes. No required check or branch rule is overridden. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
81ecaf0654 |
fix(server): bound failed lifecycle verification (#18383)
Failed lifecycle verification could allocate another test environment every minute indefinitely. Count failed host verification tests separately from claims, persist a repair wait for confirmed configuration failures or three generic failed tests, and clear it through authorized retry or execution configuration changes. Preserve typed temporary execution-target failures through adapter checks so timeouts, transport failures and interrupted probes keep automatic recovery. Retain existing readiness, authorization, termination and cleanup gates. Document the policy and add real PostgreSQL persistence and actual OpenCode adapter regressions. Verified 398 focused tests (one Windows-only test skipped on macOS), repository-wide typecheck and build, structural checks, fully green current-head CI and Greptile 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
875d763c1d |
fix: back off expired deadlines during domain drain
Keep a future in-memory resume wake when domain dispatch is paused independently of coordinator admission. Preserve the saved obligation without rescheduling its expired deadline into an immediate skipped-attempt loop. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
e3322c4a62 |
Merge remote-tracking branch 'origin/master' into codex/sleep-run-task-recovery
* origin/master: fix(evals): support official TestOps job metadata (#18385) # Conflicts: # server/src/__tests__/managed-skill-content.test.ts |
||
|
DottaandPaperclip
|
65602133e7 |
perf(heartbeat): let the dispatch worker own startup retry promotion
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2955e0e22e |
fix(server): scope run retention hints to affected companies
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
15cc12dba6 |
fix(heartbeat): scope and batch dispatch wakeups by owner
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8595efc20b |
fix(evals): support official TestOps job metadata (#18385)
## Thinking Path > - Paperclip manages AI agents and verifies real work through Product E2E tests. > - The fixed Chat feature workflow uses TestOps to select and report one backlog case. > - Official registration exposes a workflow-query URL and a Branch parameter. > - The existing reader accepts only workflow-file or numeric URLs, and the admin launcher requires an operation parameter. > - These assumptions reject valid official metadata before the feature can run. > - This pull request supports the exact observed form while preserving source, selection, and once-only execution checks. ## Linked Issues or Issue Description Refs: #18322, #18375. **What happened?** The second zero-model bootstrap stopped with a generic job-mismatch code. Ordinary TestOps UI inspection then showed an official workflow-query URL outside the accepted literals and a Branch-only job. The exact first failed predicate in that bootstrap remains unknown. **Expected behavior** Accept that exact fixed-workflow URL. Preserve the existing master Branch value when the job has no operation parameter. Keep rejecting foreign URLs, unknown parameters, wrong branches, and conflicting operation IDs before mutation. **Steps to reproduce** Pass the official query URL and Branch-only metadata through the existing bootstrap, selection, and admin-launch owners. The new pure controls exercise those actual helpers without a CLI, backend, or model call. **Paperclip version or commit** This change starts from `0e15c92073a09e50df99190a37f69a86286b989b` after #18375. **Deployment mode** GitHub Actions and the protected TestOps reporting environment. ## What Changed - Share one exact URL predicate across bootstrap, evaluation selection, and admin preparation. Add only the observed official workflow-query literal. - Support Branch-only, operation-only, and Branch-plus-operation jobs with strict definition IDs, names, values, and duplicate checks. - Preserve the exact master Branch value. Set an existing operation parameter to `evaluate`; otherwise use the fixed workflow's existing default. - Use the same checked parameter vector for job and plan writes and their readbacks. - Report distinct fixed bootstrap predicate codes without response values. - Document when the protected operation-parameter ID is required. - Align one existing Cloud-maintenance test with the inherited managed-Cloud S3 policy. Preserve the same-version local-storage proof in the parent, use valid synthetic Cloud settings in the child, and check the expected provider mismatch without S3 requests. Retain the no-seed and unchanged persisted-row assertions. ## Verification - URL controls: three failures on the parent; all pass with the correction. - Branch controls: two failures and one correct rejection on the predecessor; all pass with the correction. - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs`: 40 owning pure/mock controls pass on Node 26.4.0. - Two MJS syntax checks and `git diff --check` pass. TypeScript noEmit was not run for this MJS/documentation change. - The first [normal CI run](https://github.com/paperclipai/paperclip/actions/runs/41131094974) failed only in the managed-skill Cloud fixture and its dependent verification gate: 46 jobs passed, two failed, and one skipped. The tested merge included a new upstream policy that rejects the test's local-disk Cloud configuration. The exact caught runtime exception remains unknown. - Three pure controls using the actual policy validator pass: the old invalid Cloud fixture rejects, the new synthetic S3 fixture admits, and an invalid master key rejects. An initial tsx wrapper IPC permission error executed zero controls and remains a separate setup failure. - The PostgreSQL/maintenance-child test was not run locally. On new-head AWS CI, `managed-skill-content.test.ts` passed all 17 tests in server shard 9. - [Normal CI](https://github.com/paperclipai/paperclip/actions/runs/41133876636) is green on exact head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57`: 48 jobs passed and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed this exact head at 5/5 with no actionable findings or unresolved review threads. - No actual CLI, backend, model, provider, app, database, build, or bootstrap was run to validate this correction locally. ## Risks The exact original bootstrap predicate is still unknown. This fixes a demonstrated metadata false negative and preserves the prior failed run. Actual backend parameter types and integration dispatch still need verification. Unknown metadata stops before the once marker and writes. This PR changes no workflow, credential, case, model, grader, clock, or private-evidence policy. It does not claim a passing feature result. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
2bdda5dcf2 |
fix: release settled deadline work holds during drain
Preserve dirty generations with an in-memory resume deadline while allowing settled transaction holds to end during drain. Keep unresolved database intent fenced by the coordinator. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
bd5a104c41 |
fix(agents): revalidate error-clear authority before write intent
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9466bca1ad |
test(skills): verify Cloud CLI reads through shared object storage
Use the real S3 provider with a controlled HTTP fixture and a stable test key. Preserve read-only/no-seeding behavior and verify corrupt, missing, and restored object outcomes under managed Cloud policy. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d820f2bae4 |
refactor(recovery): expose scoped issue recovery obligations
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
50217ad02a |
test(heartbeat): use test-only signal setup and inspect saved stdout
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
098111c2ca |
test(heartbeat): retain process output and lifecycle acceptance evidence
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
b57f5e5869 |
Merge remote-tracking branch 'origin/master' into codex/sleep-run-task-recovery
* origin/master: fix(config): require shared storage for managed cloud (#18350) feat(skills): make managed content independent of web-server disk (#18334) Add a portable agent question and answer resumption journey (#18346) fix(ci): isolate test selection and require current CI CODEOWNERS (#18376) fix(evals): capture TestOps registration after watch (#18375) fix(runner): allow session reset after confirmed task retirement (#18373) fix(evals): batch task search result readback across renders (#18370) fix(ui): keep task plugin views available for manual access (#18363) test: wait for the evidence expiry worker to settle (#18241) |
||
|
DottaandPaperclip
|
922ae50f3f |
test(heartbeat): verify dispatch against real PostgreSQL clocks
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
198218bf55 |
fix(config): require shared storage for managed cloud (#18350)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed Cloud instances must retain shared objects and secrets across server replacement. > - Implicit storage defaults can select local disk or an unintended bucket. > - Cloud runtime settings now come from a canonical database snapshot. > - This pull request validates bootstrap prerequisites before database setup and effective S3 settings after snapshot resolution. > - The benefit is an actionable refusal before local signing state or HTTP setup, while valid canonical Cloud settings work. ## Linked Issues or Issue Description **What happened?** A managed Cloud instance could start with local disk storage, an implicit bucket, or a local key file. After canonical Cloud configuration landed, checking only environment or file values would also reject a valid database bucket. Shared schema defaults could make an omitted raw bucket appear explicit. **Expected behavior** Managed Cloud requires external PostgreSQL, a stable environment master key, and effective S3 settings with a nonblank bucket. The canonical snapshot remains the configuration authority. Invalid configuration refuses startup before signing or HTTP effects. Self-hosted defaults remain compatible. **Steps to reproduce** 1. Select managed Cloud with its tenant token or managed configuration document. 2. Leave storage as local disk, or omit the S3 bucket from a raw canonical snapshot. 3. Start the server. Startup must refuse. An explicit valid bucket in the canonical snapshot must work without a file or environment bucket. **Paperclip version or commit** Current base: `4edd10804e2b42941aa2038e90c362c13a0b98dc`. **Deployment mode** Managed Cloud. Ordinary self-hosted defaults remain unchanged. Related PRs: #18335 owns canonical Cloud runtime configuration. #18327 owns S3 run logs. #18352 owns credential persistence. #18347 owns cutover checks. This PR adds configuration prerequisites to those existing boundaries. ## What Changed - Validate external PostgreSQL and a stable environment master key at the Cloud bootstrap boundary, before database setup. - Validate effective S3 settings after canonical snapshot publication and before Cloud signing, runtime consumers, or HTTP setup. - Require a nonblank bucket in the Cloud snapshot schema. Reject omitted or blank buckets in every legacy Cloud file before one-time import normalization, even when an environment bucket is supplied. Keep shared self-hosted schema defaults. - Preserve typed bucket provenance at the existing file-read boundary. Accept explicit canonical bucket names, including `paperclip`, and environment settings when the snapshot is null. - Validate effective legacy storage with the existing resolver before seeding. A rejected candidate leaves no canonical rows, so a corrected source file can be imported on retry. Environment overlays are checked but never persisted. - Add real PostgreSQL snapshot and migration-retry tests, startup refusal tests, and environment/file compatibility controls. Document that Cloud always requires its stable environment key, even with an external default secrets provider. ## Verification - Latest focused acceptance passed 199/199 in six files (54.29 seconds), using the command below plus `server/src/__tests__/cloud-configuration-migration-postgres.test.ts`. Real PostgreSQL proves rejected storage leaves all three migration authority rows absent, then a corrected legacy file imports on retry. An external-default-secrets control proves the deliberate unconditional Cloud master-key contract. - Before the peer correction, the frozen product diff passed 178 tests in five files: managed-cloud-prerequisites, cloud-runtime-config, server-startup-feedback-export, config-file, and cloud-configuration-readers. Command: `pnpm exec vitest run server/src/__tests__/{managed-cloud-prerequisites,cloud-runtime-config,server-startup-feedback-export,config-file,cloud-configuration-readers}.test.ts --maxWorkers=1 --no-file-parallelism`. Real PostgreSQL used a qualified POSIX test-only PG18.6 override because the host cannot supply the stock runtime's SysV semaphore allocation. - Coverage includes raw database and legacy omitted/blank bucket rejection, valid database and environment buckets, null snapshots, explicit default-named buckets, self-hosted compatibility, and refusal before local signing mutation. - The real AWS SDK configured-provider control uses a simulated loopback S3 service. It writes, reads from a fresh provider, and deletes an object. It also reads an encrypted credential with the stable environment key. This does not qualify production cloud permissions. - `git diff --check` passed. Final merges add unrelated master changes from #18241, #18363, and #18370; the focused storage product diff is unchanged. - Peer review reproduced a fresh legacy local-disk import that gained the default bucket before an S3 environment overlay. The importer now requires a raw explicit bucket for every legacy Cloud file. Six controls cover omission/blank values with and without an environment bucket; the full focused rerun passed 184/184 across all five files in 194.89 seconds. - Initial revised startup testing passed 167/168: one fixture omitted the database URL and failed before its intended key assertion. The corrected final run passed 178/178. An initial direct server typecheck failed because Runner declarations were not built; this is a setup failure, not a compile pass. Integration verification rebuilds dependencies before full checks. - Prior head `fae1e4aa` passed complete CI with 53 successes, two intended skips, and Greptile 5/5. `de71345f379cecb71dbfb219f35e14f942c10a3f` then passed complete CI (55 successes, three workflow skips, and a neutral external review), with Greptile 5/5 and zero unresolved threads. The external review identified the preseed rejection problem; its correction is committed at `75e04d0d77e19dcef0a8cb070b6b518f0f173b14`. The final merge head is `19b809ea2823a6a4e5ceb48ef2c2bac4721733d6`; [Fresh complete CI](https://github.com/paperclipai/paperclip/actions/runs/41127309762) passed on that head: 55 successful checks, three workflow skips, and one neutral external review. Greptile reviewed this exact head at 5/5 with no actionable issues. Both earlier external review threads are resolved with the tested fixes. Earlier infrastructure retries remain documented in [the first diagnosis](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6529964690) and [the runner shutdown evidence](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6530364769). Earlier missing SDK build and fixture metadata failures were repaired before those passes. - Current frozen-head CI first attempt failed one unchanged Antigravity authentication success case in Runner Vitest 1/2 (`outcome=failure`, 1994 tests passed). The same file passed 11/11 locally in 2.33 seconds, and identical Runner source passed the prior complete CI run. Attempt 2 retried only failed jobs (the Runner shard and dependent aggregate). Both passed without a source change. No second retry was used. The exact timing cause is not established. [Failed job](https://github.com/paperclipai/paperclip/actions/runs/41127309762/job/124309181778). - No production data, buckets, or live credentials were used. Complete local repository suites are not duplicated in this worktree; current-head required CI supplies the full suite gate. ## Risks - Managed deployments with local disk, omitted buckets, file-only keys, or missing external database URLs now refuse startup. Operators must supply valid prerequisites before rollout. - Required deployment environment overlays must remain stable across replacements. For example, a saved local-disk provider with an explicit bucket can use an environment S3 provider; removing that required overlay correctly refuses startup until the environment or canonical configuration is corrected. Environment settings are never silently copied into the canonical row. - Effective legacy storage must also pass before first seeding. A bad candidate does not create authority rows. Existing saved rows remain authoritative and can require environment overrides or a canonical configuration change if they are invalid. - A present legacy Cloud file needs an explicit raw bucket even if its saved provider is local disk or its environment supplies a bucket. Correct the file before one-time import, or use the existing absent-file/null-snapshot environment-only setup. Environment overlays are not persisted as operator choices. - A previously stored canonical nonblank `paperclip` bucket is authoritative. Prior normalization erased whether that value was originally explicit. This change cannot recover that provenance and does not reject existing nonblank canonical values. - Validation proves configuration shape, not cloud permissions, database reachability, or key compatibility with existing encrypted data. Keep the master key stable across replacements. - Cloud snapshot JSON parsing is stricter; omitted or blank raw S3 bucket values need correction. No database migration or provider operation changes are added. These prerequisites do not prove that every server persistence path is disposable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`. Context window: unknown. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0e15c92073 |
feat(skills): make managed content independent of web-server disk (#18334)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use installed skills and exact version pins during execution. > - Managed skills still depended on mutable directories on the web server. > - A replacement server could lose acknowledged edits or remove an unassigned skill whose directory was absent. > - This pull request makes immutable versions the authority for managed content and installed repository reads. > - The server can rebuild delivery files from durable content after replacement. > - This establishes skills readiness for ephemeral web servers. General server draining and workspace recovery remain separate work. ## Linked Issues or Issue Description **Subsystem affected** Skills services, shared contracts, database receipts, Skill Studio, and runtime delivery. **Problem or motivation** A fresh web server must discover, read, edit, and deliver protected installed skills using the same database, object storage, secrets, and supported application artifacts. It must not need its predecessor's files or process memory. **Proposed solution** Use `managed` ownership with a null source locator. Read installed content by immutable version. Prepare packages before a short publication transaction. Persist execution selections before materialization. Recover exact pinned runtime assets and reject deletion while active or recovering pins exist. **Alternatives considered** Reconstructing a mutable installed directory preserves the old source-of-truth ambiguity. Permanent dual writes create two competing authorities. This change uses one managed model after a bounded legacy conversion. **Roadmap alignment** This improves the existing Skills Manager, Studio, version history, and Cloud execution capabilities. The maintainer supplied the implementation contract. This does not add a separate skills product or deploy fleet changes. Related work: #18082 supplies immutable package storage. #17273 addresses edits outside the API for linked local skills; this PR keeps that source category separate. #2648 is a closed earlier ephemeral-filesystem proposal. ## What Changed - Add managed ownership and focused content publication and execution materialization modules. Keep the archive format and package storage tables. - Publish managed create, save, delete, fork, rename, restore, and catalog content with version checks. Add durable create receipts. Preserve file-save receipt responses. - Return file version identities. Keep Studio drafts after a conflict and provide an explicit reload action. A delayed reload cannot overwrite newer typing or navigation. - Persist exact runtime selections, including empty selections. Rebuild pinned assets and verify their admitted digests on a replacement server. - Preserve source access on forks and disconnected copies. Keep the existing workspace policy category for managed skills. - Add bounded conversion, eligibility, and independent verification phases. Load persisted Cloud configuration in the standalone CLI without seeding or changing it. Reject partial current versions against installed inventories; recover only complete saved content or the exact source. Document storage migration blockers and the compatible rollback floor. - Add separate-process PostgreSQL/S3 qualification and update domain, retention, policy, and UI regressions. - Pass the plugin worker manager into Studio execution, admit restricted `skill_test` mode through native Runner contracts, and preserve admitted scope across native and REST tools. Native tests cannot create ordinary tasks, browse company data, or use external tools. - Select the exact durable Studio revision even when unassigned or Beta skills is off. Persist Studio output and audit atomically with native task completion, preserving the result on replay. ## Verification - Final revision `13e8af480b88ba6fea5737bac4bd81104d604792` passed full CI: all 49 workflow jobs completed successfully ([run 41127396098](https://github.com/paperclipai/paperclip/actions/runs/41127396098)). This includes typecheck, test suites, Runner checks, browser checks and build. Greptile review 15 is 5/5 on that same revision, with zero unresolved threads. The PR is ready for review. - Local workspace typecheck and build passed after integrating master. Focused checks passed: 33 execution/completion cases, 52 Runner HTTP cases, 16 managed-content cases, 17 Studio cases, the new real-process CLI regression, map checks and UI token gates. The CLI regression proves inspection does not seed missing Cloud configuration. Server typecheck passed after that fix. - The independent-process PostgreSQL/S3 matrix passed through normal HTTP skill routes. It covers A → B → C replacement, lost commit responses, upload crashes, concurrent saves, binary contents and executable delivery. Metadata listing and warm selected delivery retain zero object downloads. Local timing measurements are recorded; they do not establish hosted latency. - Hosted A → B → C passed on the owned disposable test tenant with external Postgres, S3 and a real remote Paperclip Computer. Predecessor temporary homes were absent. C read and executed B's acknowledged managed save. New runs used repository v1 while v2 was installed. Hashes matched an independently checked exact upstream commit. - Compatible image rollback to `6518b323` passed. Studio retained the unassigned fork's C save. A fresh remote run executed the newer managed content and historical repository pin. The actual shell event confirmed expected hashes, stdout, exit 0 and mode 0555. - Runtime candidate `9200e3fcb4` deployed successfully ([Cloud run 41124147876](https://github.com/paperclipai/paperclip-cloud/actions/runs/41124147876)). The final Studio test delivered only the unassigned fork's exact v3, executed its script and persisted a successful output snapshot. Independent command-event inspection confirmed all three hashes, stdout, exit 0 and mode 0555. All 14 installations were version-backed. Independent verification checked all 25 versions: 23 remote objects, two bundled database snapshots, zero blockers. - The deployed image is `9200e3fcb4`. Later master integration, the client-only delayed-reload draft guard and the CLI bootstrap have separate automated proof. The CLI bootstrap was also exercised against the tenant's stored configuration and storage through the same read-only services. This PR does not claim that the deployed image is the final PR revision. - Live Studio checks exposed worker wiring, admission, access scope, exact selection and completion-publication gaps. They are fixed. Failed attempts remain in the evidence. The full local suite exceeded 80 minutes under host contention and was stopped. Later combined/isolated local attempts also timed out; they are not counted as passes. Full CI covers the integrated suites. Earlier CI found an obsolete restart-scope fixture, which was corrected and passed all 52 HTTP cases. - Reproduction and migration: [`doc/skill-ephemeral-web-servers.md`](doc/skill-ephemeral-web-servers.md). Source revisions, deployments, versions, run IDs, hashes and failure history: [`doc/plans/2026-10-30-ephemeral-web-skills.md`](doc/plans/2026-10-30-ephemeral-web-skills.md). ## Risks - Legacy conversion requires stopped admission and retired filesystem-authoritative writers. Pre-conversion binaries are not supported rollback targets afterward. - Cloud readiness requires remote object storage. Local provider receipts and provider mismatches remain blockers until objects are copied and verified. - Linked local sources retain filesystem semantics and are outside the cloud guarantee. - Cold reads still fetch the existing archive format. Warm runtime preparation retains the verified-cache path. - Bundled reconciliation preserves an installed head. Existing release snapshots remain selectable. An older server must not replace a newer head with its shipped files. - No production volumes, fleet defaults, or general deployment coordination change in this PR. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser use. The exact backend model ID and context-window size are not exposed in this session. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
6174b4d7c8 |
fix(runner): allow session reset after confirmed task retirement (#18373)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native warm sessions retain a Runner owner between turns. > - Closing that owner must stop execution and preserve valid recovery state. > - A failed suspension can leave a rejected close promise after the task has stopped. > - A session reset then retries that same promise and blocks every fresh session. > - This PR separates task retirement proof from checkpoint reuse for a new session. > - A reset can proceed after exact stop proof, completed handoff, and file collection. ## Linked Issues or Issue Description Refs #18011. That change preserves checkpoint authority during warm preparation. This change handles a stopped owner whose checkpoint cannot be reused. Related: #14695 covers managed files retained with warm processes. A search found no duplicate stopped-owner reset fix. **What happened?** A warm Runner can stop and complete its task handoff while close reports `runner did not durably suspend before checkpoint`. The transport caches that rejected close promise. Core retains the warm owner. Each session reset retries the same promise before it can launch a provider. A controller restart clears the in-memory owner and permits recovery. **Expected behavior** A reset to a new normalized session should release the old owner when its exact task has proved that execution stopped and its handoff completed. Core must collect stopped instruction files before it admits the new session. An unconfirmed owner must continue to block replacement. **Steps to reproduce** 1. Complete a warm native run through a plugin task provider. 2. Change configuration so Core retires the warm owner. 3. Let the task stop and complete, but fail durable suspension proof. Close keeps its rejected promise. 4. Reset the session and start another run on the same task environment. 5. Observe the old close error before any new provider work starts. ## What Changed - Keep a lease-bound retirement receipt in the plugin task owner. It requires `executionStopped: true` and successful task completion after credential handoff. - Accept an unrecoverable suspension error only during retirement for a different normalized session, with that exact owner's receipt and settled Core tool results. - Collect stopped instruction files before releasing the warm owner. Preserve old checkpoint evidence. - Keep same-session, active-owner, incomplete-handoff, unrelated-owner, unresolved Core tool results, and other close-error fences. - Report Core tool settlement on the close error after ingress retirement and drain. Default missing proof to false. Expose the error through the server's Runner shim. Add regression tests and document the recovery boundary. ## Verification - The reset regression fails with the new handling disabled and passes with it enabled. - Both affected suites passed: 1,032 tests. The final 15 focused owner/reset cases pass after adding the tool-settlement guard. - The assembled regression uses the real task-owner and backend binding. Its controlled task provider proves stop and completion. Close returns the same rejected promise on every call. - Negative cases cover incomplete handoff, other close errors, same-session continuation, provider change with the same session ID, unrelated receipts, unresolved Core tool results, and failed file collection. Collection can later succeed without replaying stop or reading the old checkpoint. - Five real-process transport cases pass. They cover an insufficient suspension deadline, the cached rejected promise, late tool completion within and beyond the deadline, interruption, and result-persistence failure. Unsettled callbacks produce `semanticToolsSettled: false`. - `pnpm -r typecheck` passed with the repository's pinned Rust toolchain on PATH. After the final contract change, Runner TypeScript build and server `tsc --noEmit` passed. - The local full-suite run was stopped at the requester's direction. Full regression and build coverage are delegated to GitHub CI. - Full GitHub CI passed on `ef0040d80af679ce6d4a130468e15f75678c1f4c`. Standard Greptile reviewed the same commit: 5/5, zero comments. Apex then reviewed that same commit: 5/5, zero comments. Final checks confirm no unresolved review threads and no merge conflicts. - Initial full CI on `00c002fd5a` failed one scheduled-retry route test at 18/20 cancellations. Controlled cancellation latency reproduced the short poll-budget failure. Before the next cycle, master advanced with #18322, which coordinates that test at the first completed batch. The separate local timeout commit was dropped during rebase. The upstream test passes locally. The rebased run passed 46 targeted owner/reset and scheduled-retry route cases. This PR has no scheduled-retry source or test diff. - No deployed provider run or tenant rollout is claimed by these source tests. ## Risks The exception applies only to a new normalized session and an exact completed plugin task. The old checkpoint remains evidence and cannot authorize reuse through this path. The original suspension failure can remain visible on its run. This fix prevents repeated startup failures after a reset; it does not hide unconfirmed cleanup or make every suspension succeed. Missing task receipts and failed file collection continue to block replacement. No schema migration, plugin protocol change, Rust Runner binary change, or new polling is required. Tests use a controlled task boundary; they do not certify every external provider implementation. ## Model Used OpenAI Codex, GPT-6 family, with tool use and local code execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal issue or instance details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Greptile Apex is 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
294d8ca1c2 |
perf(heartbeat): schedule retry promotion from durable deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
fd008979f5 |
test: wait for the evidence expiry worker to settle (#18241)
## Thinking Path > - Paperclip coordinates AI work and retains execution evidence. > - Evidence retention uses an asynchronous worker and a shared scheduler. > - The retention test checked the next deadline as soon as an archive disappeared. > - File removal can finish before the worker publishes that deadline. > - This pull request waits for worker settlement before asserting its next wake. > - The test keeps its deletion, busy-session preservation, and quiet-worker checks. ## Linked Issues or Issue Description **What happened?** CI for #18239 failed in `native-harness-backup-expiration.test.ts`. The healthy-expiry test observed the file removal while `nextWakeAt()` was still null. Subtracting the current time then produced a large negative number. **Expected behavior** The retention test should check the deadline after the asynchronous worker has settled. **Steps to reproduce** Run the server test shard with a slow filesystem completion. The deletion predicate can become true before scheduler publication finishes. ## What Changed - Keep the existing file-removal wait open until tracked work is idle and the expected busy-session retry deadline is registered. - Keep the held archive and quiet-worker assertions. ## Verification - Original CI failure: [server shard 3/12](https://github.com/paperclipai/paperclip/actions/runs/40976031169/job/123804262266). - The targeted test passes with its real PostgreSQL fixture: `node node_modules/vitest/vitest.mjs run --project @paperclipai/server server/src/__tests__/native-harness-backup-expiration.test.ts -t 'lets healthy saved expiry preempt' --maxWorkers 1`. - After rebasing onto master, the focused retention test passes again with its real PostgreSQL fixture. The PR now changes one test, with seven insertions and two deletions. - Master independently fixed the authentication timeout and GitHub-recovery checkpoint race. Their redundant commits were dropped during rebase. - [Full GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/40984842413) passes on `bbaa7c3c384f6c220ce2c979f0e9b5d01ddb1f77`: all 48 jobs succeeded. Standard Greptile is 5/5 on that head, and GitHub reports the PR clean. - Broader coverage runs in GitHub CI. No local full build or full test suite was run for this assertion-only change. ## Risks Low risk. The retention test waits for the condition it already asserts. The existing deletion, busy-session preservation, future-deadline, and quiet-worker checks remain in place. No production code or time budget changes. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, terminal tools, and code execution. The serving variant and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
76fde891cc |
Add bounded recovery file inventory before server cutover (#18347)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Accounting, credential recovery, and sandbox cleanup can retain legacy files on an instance volume. > - Replacing that volume while files remain can lose unresolved work. > - There is no verified production backlog, so an empty installation needs no migration. > - The recovery owners must decide whether each record is valid and settled. > - This pull request adds a bounded file inventory and an optional cutover gate. > - The benefit is a safe check that preserves obligations without reading secret material. ## Linked Issues or Issue Description **Problem or motivation** Operators need to check the actual affected instance directories before replacing a web server. A skipped corrupt file or failed read must not appear clear. **Proposed solution** Add a Node script that inspects only the three legacy recovery directories for an explicitly chosen instance. Default mode reports counts. The optional check mode blocks when files or uncertainty remain. **Alternatives considered** A bulk migration system has no established backlog to migrate. Reimplementing the owner parsers could expose credentials and still would not prove settlement. This check never reads file bytes. **Roadmap alignment** This is a small operational prerequisite for safe cloud server replacement. It does not add a recovery backend or change resource ownership. Related recovery-owner changes: #18354 (accounting), #18352 (credentials), and #18357 (sandbox ownership). This inventory remains independently usable with existing files. ## What Changed - Add a pure classification policy and a bounded filesystem boundary. - Add `node scripts/recovery-cutover.mjs --instance-root /absolute/instance` with optional `--check`, directory override, and entry budget. - Report observed lower-bound counts and safe status summaries. Do not print paths, filenames, identifiers, raw errors, or file contents. - Add one dependency-free CLI acceptance suite and a Vitest bridge so the same cases run in normal server CI. - Document existing owner entry points, override selection, stopped producers, and retained-volume requirements. ## Verification - `node --test scripts/recovery-cutover.test.mjs`: 9 cases pass. The suite starts the actual CLI with synthetic legacy directories. - Pending receipts, nested credential files, orphan records, malformed bytes, temporary files, unreadable files, unsafe paths, symlinks, read failures, bounds, and configured overrides all block safely. - Empty or absent recovery directories clear. Source file bytes remain unchanged. Report-only mode and check-mode exit codes are tested. - `pnpm exec vitest run server/src/__tests__/recovery-cutover.test.ts --pool=forks --maxWorkers=1`: the normal CI bridge passes and runs the same 9 cases. - Cached frozen dependency installation, script syntax checks, and `git diff --check` pass. The lockfile is unchanged. - Independent source review found the read-only boundary sound. - [Complete current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41097614666) passed at `bf5cc7f067041c10cd0294ff26756678da943976`: 53 successful checks and two intended skips, including typecheck, full tests, build, and E2E. Greptile is 5/5 with no review threads. - One failed-job retry was needed for an existing heartbeat timing failure in server shard 4. The relevant heartbeat source and tests match the base branch. Local diagnostic runs had 61 pass/2 fail and 7 pass/1 fail, with changing timing failures; neither run is claimed as green. The bounded retry and all failed evidence are [recorded here](https://github.com/paperclipai/paperclip/pull/18347#issuecomment-6529629804). No heartbeat code or test was changed. ## Risks - The scan is a point-in-time inventory. Stop producers for the final gate. It is not an atomic drain certificate or protection against hostile path mutation. - All files remain unverified. The check cannot certify validity or Postgres settlement. Existing owners retain that responsibility. - The scan does not inspect other instances, volumes, Runner delivery, or database obligations. It does not establish that production cutover is ready. - No migration, deletion, decryption, or replay command is added. Symlink paths are rejected, including system aliases; use canonical absolute paths. ## Model Used - OpenAI `gpt-6.1-sol`, with reasoning, tool use, and code execution. Context window size was not supplied. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |