28 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	f6ba9afab0	perf(server): narrow explicit continuation run reads (#17896) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Recovery must prove that old execution stopped before it admits new user input. > - Explicit continuation reads the source run before and after it takes the run lock. > - Both reads transfer the saved prompt and the full Runner profile. > - The preflight needs only identity, comment, adapter, and workspace receipt fields. > - This pull request narrows those reads and keeps the complete locked profile for session proofs. > - The benefit is less database transfer with the same admission and recovery rules. ## Linked Issues or Issue Description **What existing behavior does this improve?** Explicit native continuation admission after a failed or stopped run. **Current behavior** The preflight and locked reread select complete heartbeat run rows. Large saved contexts and execution profiles cross the database connection twice. **Proposed behavior** Read only required scalar fields and typed context keys. Read only historical adapter identity and workspace sync receipts before the lock. Keep the complete Runner profile in the locked reread. Keep lock order, company scope, text lookup of historical run IDs, stop proofs, queued input, and replay rules. **Reason and benefit** A synthetic PostgreSQL 17 fixture removes 74.9% of the two queries' DataRow bytes. This result is local query evidence. It is not a claim about deployed latency or monthly transfer. **Breaking changes** None intended. No schema, API, migration, workflow, or dispatch contract changes. **Additional context** Related: #17491 narrows the separate execution blocker read path. This PR covers explicit continuation admission. The PR has no dependency on the other byte reduction PRs. ## What Changed - Select required continuation fields and typed context keys instead of complete run rows. - Read a small historical adapter/workspace profile during preflight. Preserve the full locked profile for exact suspended-session and quarantined-cleanup proofs. - Narrow four helper input types to their actual field requirements. Their runtime behavior is unchanged. - Add real PostgreSQL admission comparison coverage and a reproducible baseline/treatment benchmark. - Document the query boundary and benchmark command. ## Verification - Baseline: `663fe667d66d3863d0919a8724f1d99500feef0b`. - Disposable PostgreSQL 17.11. One source run has a 192 KiB incompressible context and two 96 KiB profile payloads. - Command: `pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-continuation-run-reads.ts --baseline-ref=663fe667d66d3863d0919a8724f1d99500feef0b`. Set `PAPERCLIP_CONTINUATION_BENCHMARK_DATABASE_URL` to a disposable database ending in `_test` to use an external server. - The harness captures SQL generated by both real service versions. Baseline imports use current helpers; only their TypeScript input types changed in this PR. Their runtime behavior stays fixed. - Preflight DataRow bytes: 394,067 to 466 (99.88% less). Locked reread: 394,067 to 197,168 (49.97% less). Combined: 788,134 to 197,634 (74.92% less). - Byte counts cover exact PostgreSQL text-format DataRow frames. They exclude RowDescription, command metadata, TLS, TCP, and other queries in the service. - Seven measured samples per query follow two warmups. The order alternates. Final-head median preflight time: 6.53 to 2.28 ms (ranges 5.34–7.41 and 1.84–4.46 ms). Median locked read: 8.06 to 5.26 ms (ranges 5.94–13.84 and 4.61–6.81 ms). These are warm local SQL measurements under shared machine load. The deterministic byte counts are the acceptance criterion; these noisy timings do not establish a deployed latency gain. - The harness compares the admitted continuation, blocked reasons, every retained scalar column, kept JSON, and the full locked profile. It fails unless the fixture removes more than half of the two reads' DataRow bytes. It records SQL, parameters, revision identity, samples, and EXPLAIN plans. - All 291 continuation tests pass together on the final head with no skips in a clean isolated PostgreSQL 17 database. This includes all seven new projection comparisons and six malformed/missing JSON cases. - Edited server TypeScript check passes: `node node_modules/typescript/bin/tsc --noEmit -p server/tsconfig.json`. - `pnpm -r typecheck` and `pnpm build` pass on the final head. The normal server build verifies the complete pinned Copilot closure. Local full `pnpm test:run` is not claimed: host embedded PostgreSQL startup limits prevent that gate. The isolated PostgreSQL regressions pass; the full exact-head test matrix runs in CI. The targeted exact-head test retry passes all 4,086 tests in 98 files. All CI verification checks and the dependent aggregate are complete and successful on head `47c1b5f7d17a58310f7a3ebd494fdcaa814777c2`. Greptile completed cleanly on head `47c1b5f7d17a58310f7a3ebd494fdcaa814777c2`: 5/5, no findings, and no unresolved review threads. - Local setup evidence: embedded initdb could not load its ICU data library. Tests therefore use a dedicated PostgreSQL 17 container. A run against reused test fixtures produced stale sweep failures; a new clean database passed all 285 cases. The first comparison mock replaced an unrelated aliased query; it was narrowed to the two admission reads before the clean pass. - The first [server test shard 6/12](https://github.com/paperclipai/paperclip/actions/runs/40529210562/job/122277218565) failed one existing workspace-recovery case: a third export stayed `scheduled` at its 20-second poll deadline; 4,085 other tests passed. The exact case passes on the pinned baseline and final head with the same PostgreSQL 17 fixture. Temporary spies recorded zero calls to both changed admission functions in either replay. Other touched helpers have type-only changes. This supports one timing-flake retry; the exact scheduler delay remains unproven. The single [targeted shard retry](https://github.com/paperclipai/paperclip/actions/runs/40529210562/job/122285744135) passes all 4,086 tests in 98 files. The exact case completes in 17.25 seconds. The dependent aggregate is complete and successful. No timeout or unrelated runtime code was changed. - The separate Paperclip Review check ended `NEUTRAL` without a final report. The completed source review is the exact-head Greptile 5/5 result above. ## Risks - A future admission check may need an additional saved context key. Keep the projection and helper contracts in sync with that consumer. - Exact suspended-session and quarantined-cleanup proofs still require the complete Runner profile. This PR retains it under the lock rather than weakening those checks. - JSON values retain their types and missing keys stay absent. Malformed-context comparisons preserve the existing controller hold. - These measurements use synthetic local data. Deployed transfer savings require a fresh capture after rollout. Nothing was deployed. ## Model Used OpenAI GPT-6 through Codex. The exact runtime model ID and context window are not exposed by this session. The work used repository tools, code execution, and real PostgreSQL validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 12:05:51 -05:00
DottaandPaperclip	ac22b1b1f9	fix: preserve recovered input and task directory continuity (#17878) ... <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/60/changes/paperclip/60 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Recovery must preserve files and the authority of each human request. > - Some failed native runs preserve a snapshot without completing native copyback. > - The saved input could remain blocked after recovery settled. > - A later turn could also treat the new task directory as a Git base because it inherited the agent default. > - This change continues the same authorized input once and keeps its saved directory usable on later turns and Retry after a pre-provider preparation failure. ## Linked Issues or Issue Description Refs #14905. Related: #17724 covers cleanup handoff after native copyback. This change covers snapshot-only preservation and later turns in its directory. #17814 covers a separate same-run checkpoint restore defect. The open legacy-adapter change in #13291 does not cover this native saved-input path. ## What Changed - Forward the trusted deferred inbound request to native continuation admission. Verify the exact unconsumed receipt and its source, actor, company, agent, and task. - Let historical holds for the same failed run use its settled recovery strategy. Keep each controller and termination check. - Keep the old native sync proof unchanged. Attach recovered files through the verified snapshot path in a fresh attempt. - Recognize only the current same-task, active local directory in the canonical task or attempt namespace. Keep it on later turns instead of applying inherited Git defaults. - Keep the current healthy directory when canonical Retry follows the exact pre-allocation Git-base failure. Require the committed source link, strict server-produced failure events, no execution evidence or unfinished owner, and the current canonical binding. Recognize the known configuration warning without accepting arbitrary or provider-origin warnings. - Keep owned-directory readiness separate from repository-free strategy selection. A repaired real Git checkout with explicit task/project/assignee Git policy uses normal Git validation and preparation. Preserve explicit fresh, new sources, immutable native bindings, and locked path/binding checks. - Add real PostgreSQL and filesystem regressions for admission, file preservation, and the complete later-turn heartbeat policy chain. ## Verification - Real PostgreSQL callback/queue regressions are RED on unchanged production and GREEN with this repair: one successor from the same authenticated saved request, with filesystem snapshot preservation. The 24-case source/Retry matrix passes and retains termination, allocation, generation and revocation negatives. - Actual browser use exposed both later-turn Git-base failure and canonical Retry selecting an empty sibling. The real heartbeat selection/configuration/preparation/Retry regressions reproduce each. All 81 workspace-preparation cases passed before the final repaired-Git edge; all 17 final affected phase-chain/safety cases pass, including that new case. Its finalized oracle checks effective Git policy, an actual Git checkout and saved-file bytes. Known preparation warnings pass; unknown/provider warnings and execution contradictions do not certify reuse. - Full workspace typecheck, full build with matching `5342bd52da` stamp, maps and diff checks pass. Runtime commit `5342bd52da` passed CI with 53 successful checks and 2 conditional Storybook skips, and Greptile 5/5 with zero unresolved threads. The follow-up `9096fe8f62` removes only the feature/access map additions as requested; runtime and test files are byte-identical. Map structural and diff checks pass. Greptile reviewed this exact final commit at 5/5 with no unresolved threads. All test and build jobs passed. The final required `ci / verify` and `ci / e2e` aggregate gates are queued without runners; one bounded gate-only requeue retained every successful job. GitHub auto-merge is enabled and waits for those gates. - Normal UI acceptance on exact final source with actual OpenCode, configured DeepSeek model and Daytona passes both turns: snapshot continuation writes a useful review/report; reload and a second ordinary message retain the same workspace and logical native/provider session. All six files remain byte-identical, the report's computed SHA and unpredictable marker are confirmed, the concrete result is actual 250 versus correct 350, native finalization/copyback commit, and normal lease cleanup succeeds. A separate normal UI canonical Retry on preceding `9ac1d29147` preserves the current directory/report/marker and writes a continuity note; the final Git-policy split leaves that exercised non-Git path unchanged. - Boundaries: local app/server source with real remote native execution; seeded historical failure/stop receipts and actual stored snapshot, not a replay of the original staging allocation or external review publication. Prior local verification rewrote installed dependency metadata; its generated lock diff was retained and reverted. Final launch/end source and dependency metadata hashes match; no installed dependency byte-identity claim beyond that observed freeze. - Required full local suite was attempted but remains incomplete: embedded PostgreSQL bootstrap failures prevented assertions in several suites, so only its owned test tree was stopped. A prior workspace-recovery test timed out and the following seed collided with an active hold; both unchanged tests passed in isolation. Runtime-head `5342bd52da` hosted CI passed the full gate; the documentation-only follow-up waits for the aggregate jobs noted above. Its [first Rust attempt](https://github.com/paperclipai/paperclip/actions/runs/40528918372/job/122276159874) failed an unchanged ACPX shutdown cleanup assertion; the preceding same-Rust head passed. [One bounded Rust-only retry](https://github.com/paperclipai/paperclip/actions/runs/40528918372/job/122281990961) passed. No test assertion or production guard was weakened. ## Risks - Incorrect input authority could repeat work. The existing owner reauthorizes the same durable source; admission verifies its receipt under the task lock. - Directory recognition could select unrelated files. It requires the current scoped binding and canonical namespace, then preparation rechecks the stored binding and path under the source lock. Missing or symlinked retained roots are not recreated. - Snapshot preservation does not certify native copyback or old external outcomes. The old run and its unknown outcomes stay unchanged. Unconfirmed controllers or termination remain blocked. - There is no schema migration or new public API. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, code execution, and tool use. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — targeted suites; the full local attempt limitation and final CI proof are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 16:02:43 +00:00
DottaandPaperclip	663fe667d6	fix(recovery): reconstruct missing native workspace obligations (#17877) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A failed native run can still own prepared files in a remote sandbox. > - Cleanup must preserve those files before another executor can start. > - Older failure producers can omit the preservation record. > - A saved message then waits for cleanup without a recovery owner. > - This change reconstructs that record during user continuation and orphan cleanup. > - The existing worker saves the files, verifies stop, and returns input to normal admission. ## Linked Issues or Issue Description **What happened?** A failed native run retained a prepared workspace and an active sandbox lease. It had no file-preservation obligation. Cleanup protected the files. The next user message waited for a stop receipt that no worker owned. **Expected behavior** A new message registers the missing maintenance work at once. Existing orphan cleanup recovers historical saved input. Ordinary Retry remains available. Saved input can start once, after verified preservation and containment. **Steps to reproduce** 1. Persist a failed native run with an exact prepared remote workspace and its active lease. 2. Leave its finalizer terminal and unowned. Omit the workspace recovery record. 3. Send a user comment or run orphan cleanup. 4. Observe that no preservation worker starts and the comment remains deferred. **Paperclip version or commit** Reproduced on `bd382eee536f4ada680072b1cc19100163c3d04a`. **Deployment mode** Native Runner with a remote sandbox. Local verification uses disposable PostgreSQL 18 and real temporary workspace files. Refs #17788 and #17800. Those fixes preserve future deadline failures and register historical preservation on Retry. This change adds automatic recovery for new and saved messages. Refs #17839 for the separate restart-failure producer repair. ## What Changed - Register missing preservation before an authenticated continuation waits for remote cleanup. - Reconstruct the same exact-source obligation during the existing orphan lease sweep. - Notify the existing preservation worker in the registration transaction. Do not add a timer or history scan. - Test ephemeral and reusable sources, live owners, active executors, mismatched sources, rollback, lock contention, Retry and saved-input delivery. - Document the recovery contract and feature-map behavior. ## Verification - Red: both new orphan-recovery cases fail on unchanged production code. Four ownership and source negative controls pass. - Red: a new comment starts no preservation work without the admission repair. - All 461 test cases across five affected suites pass across the local runs. One existing 30-second timing case timed out under concurrent verification; its isolated rerun passed with the same deadline and assertions. - The three final-head entry journeys pass again: HTTP Retry, HTTP comment creation, and historical saved-input recovery. - `pnpm -r typecheck` and `pnpm build` pass on final head `8fab8b2ba9049f640fd884094a2007a69465e6da`. The affected server typecheck and build also pass. - The integration journey holds the provider stop reply, checks that no successor starts, preserves real files and newer host edits, and checks one saved-input successor. - Module-boundary checks, map checks and `git diff --check` pass. - Native macOS PostgreSQL cannot start because host semaphore capacity is exhausted. Database verification uses separate disposable PostgreSQL databases for each suite. An earlier shared-database attempt caused fixture interference and is not counted as a pass. - Greptile gives 5/5 with no findings or unresolved threads on `8fab8b2ba9049f640fd884094a2007a69465e6da`. All final-head GitHub checks are settled: 55 passed, two Storybook checks skipped, and one neutral Paperclip review. Full CI test, build, typecheck, Runner and browser lanes pass. No full local `pnpm test:run` pass is claimed. The fix is not deployed. This PR makes no claim of a live hosted model turn or recovery of the original incident. ## Risks - Registration grants maintenance authority only. Existing coordinator, descriptor, allocation, stop, file-receipt and execution gates remain effective. - A locked source remains held. Its one-second lock timeout does not prevent an unrelated source from registering recovery. - New preservation writes wake existing work. Settled sources produce no new recurring work. Historical discovery uses the existing orphan cleanup lifecycle and its backoff. - Provider lifecycle is simulated in local integration. PostgreSQL, HTTP routing, file export/merge, the worker and queue admission are real. - No schema, credential, model, trust, retry-budget or assignment change. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection, tool use and code execution. The runtime does not expose the exact model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 09:50:50 -05:00
Devin FoleyandPaperclip	832c44f26a	fix: resume user follow-ups that reassign a task (#17821) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The execution control plane saves user messages while an earlier run needs recovery. > - A user comment can also assign a task back to its agent. > - That update creates an assignment wake, but continuation accepted only comment wake reasons. > - The saved question could stay blocked after workspace recovery and confirmed process cleanup. > - This pull request admits assignment wakes that carry a valid, undelivered user comment through the existing continuation checks. > - The same rule applies to new messages and saved requests already waiting for recovery. ## Linked Issues or Issue Description **What happened?** A human follow-up that also assigns a task back to its agent produces `issue_assigned`. After a native run stops during handoff, the continuation policy rejects that wake reason. The question remains saved behind the execution recovery hold even after the workspace is recovered and the old sandbox is destroyed. **Expected behavior** Deliver the saved human question once after the existing stop, ownership, authority, and execution gates pass. Assignment alone must not grant permission to continue. **Steps to reproduce** 1. Finish task work by assigning it to a human for review. Let the native run stop with `issue_reassigned` and retain its recovery hold. 2. Recover the workspace and confirm destruction of the old sandbox. 3. Post a new human question while assigning the task back to the agent. 4. Observe that the assignment wake stays deferred instead of delivering the question. Refs: #17288, #14156. These related changes cover legacy reassignment and assignments without comments. This change covers a human comment carried by an assignment wake behind a native or legacy continuation hold. ## What Changed - Share the comment-bearing wake classification between continuation admission and saved-request recovery. - Accept `issue_assigned` only through the existing persisted user-comment validation and execution gates. - Add database regressions for new and saved assignment wakes, concurrent recovery, single delivery, and unchanged prior run results. - Check that bare assignments, invalid authors and comments, changed ownership, active execution, and pending approvals or questions retain their holds. Cover native and legacy cleanup paths. - Update execution semantics and both recovery maps. ## Verification - Red: both new and saved assignment-with-comment cases stayed deferred without a successor on the original code. - Green: the final continuation suite passes all 284 tests. The four companion recovery, retry, and queue suites pass another 208 tests. - `pnpm -r typecheck` and `pnpm build` pass. - `git diff --check`, `node scripts/check-maps.mjs`, and `node scripts/check-module-boundaries.mjs` pass. - Apex review is 5/5 on `56c9c2d90dcda55b3f760946c9aba6aee448c361`, with no unresolved review threads. The only earlier finding concerned the contribution rule against new source comments; those comments were removed. - Full `pnpm test:run` is still running locally. CI is waiting for the `paperclip-public-pr-x64` runner fleet; the superseded workflow was cancelled so the current head could proceed. No passing CI claim is made yet. - Live recovery: corrected one existing saved wake to the supported comment reason. The normal server admitted one successor with the exact saved question and prior-run context, then started real tool calls. This verifies admission on the existing deployment; the new source change is not deployed yet. The resumed turn later failed with a separate provider terminal event whose error was null, followed by a workspace restore failure. The task is blocked again on that failure. Its useful answer has not been delivered, so end-to-end completion remains unverified. ## Risks - Assignment wakes now reach continuation validation when they carry a comment. The persisted comment must still belong to the task and requesting human, follow the stopped run, and remain undelivered. - The existing stop proof, cleanup, current assignment, approval, pause, budget, and competing-execution checks remain effective. Regression cases cover both native and legacy paths. - No schema migration or data rewrite is required for saved requests. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, tool use, and code execution. The runtime does not expose the exact model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 20:19:16 -07:00
DottaandPaperclip	3f1ad6b25b	fix(chat): resume task Retry after retained-source containment (#17317) ... ## Thinking Path > - Paperclip manages agent work through task and execution ownership. > - A task Retry must retain the user's request while the system verifies an old executor. > - Connected conversations also retain their external principal and accepted source batch. > - Chat Retry admission required containment before it could save a wait for that recovery. > - A saved chat receipt could also remain deferred after containment became available. > - This pull request lets the exact recovery-bound request wait and resume through its chat owner. > - Normal admission still checks containment, current access and policy before one successor starts. ## Linked Issues or Issue Description **What happened?** An ordinary task Retry on a connected conversation failed with `failed_chat_retry_not_authorized` while its retained remote executor still needed verification. The new recovery generation existed, but the chat action failed before it could save a dispatch wait. This occurred after the repairs in #17267 were confirmed live. The local regression found a second gap. A processed chat action with a deferred wake was not reconsidered after containment. Re-entering normal admission also tried to insert its existing durable receipt again. A historical hold for the same failed run could also hide the current Retry's workspace strategy, so the request still waited after the old executor was contained. Preparation could then read that sibling instead of the recovery record that owned the fresh successor, losing the fresh-workspace decision. **Expected behavior** Save the exact authorized Retry while its matching recovery generation verifies the executor. Once containment is recorded, reconsider the same chat receipt through normal admission. Start one successor with the original source batch. Keep publication, principal, pause and budget checks. **Steps to reproduce** 1. Admit a GitHub or Telegram request and record a failed native run with a retained remote lease. 2. Leave the executor's termination unconfirmed and exhaust its prior recovery generation. 3. Use ordinary task Retry to create a new recovery generation. 4. Process the connected Retry. The old admission code fails instead of retaining a wait. 5. With a deferred receipt, record exact containment and reconcile again. The old dispatch path leaves it stranded. **Paperclip version or commit** Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch is rebased onto current master. **Deployment mode** Hosted tenant for the reported failure. Isolated local PostgreSQL and the assembled heartbeat/chat services for the regression. Refs #17267 and #16965. This change does not add a Cloud dependency or change the absence protocol. ## What Changed - Permit a saved admission wait only for the exact current task Retry and active source recovery generation. - Keep promotion and execution dependent on independent containment proof. - Bind a Retry to active automatic recovery without replacing its generation, attempt count or deadline. - Share the current exact Retry strategy across historical holds for the same run. Retain each hold's stop and effect checks. - Select the canonical recovery record that proves the committed fresh successor during workspace preparation. - Let task receipt reconciliation return the same deferred action to its chat owner. - Recheck current board and external authority, then update the existing wake atomically with its successor. - Preserve the exact chat source batch. Keep generic board-comment filtering unchanged. - Add GitHub and Telegram cases for waiting, containment, one successor, stale generations, source edits, endpoint pause and supersession. - Update recovery documentation and capability maps. ## Verification - Red: real PostgreSQL tests reproduced the admission rejection for GitHub and Telegram. Six active-recovery cases then failed on the generation-ID assumption. Two same-run historical-hold cases remained deferred after containment on the old strategy lookup. A preparation regression then returned no readiness when the historical hold was stored first. - Green: 73 selected chat integration tests and 394 continuation, task Retry, workspace recovery, recovery-state and workspace-preparation tests passed. The chat tests use the real chat admission and heartbeat queue. Unrelated capacity stays occupied so no real provider starts. They assert no executor before containment, one successor afterward, unchanged source leases while waiting, preserved recovery budgets, and denial after authority changes. A separate source hold still blocks work. Both sibling insertion orders pass through the committed-readiness lookup and real workspace preparation: the new directory has independent Git storage and files, and unpublished old work remains intact. - `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff --check` passed after the review fixes. The source build passed on `f6660efa48e37a80cd9857217a537606de85b5c8`. - The local host reached its PostgreSQL semaphore limit. Successful runs used a test-only loader to reduce disposable cluster connection and worker limits. No repository database helper or application setting changed. - The full local `pnpm test:run` began before the final review edits, failed in its first server phase and was stopped after those failures. Later phases did not run. This is not a full local pass. The transient Retry-notice case passes after fixture isolation; the four Discord close-owned publication cases also pass in isolation. Other broad-run failure causes are not established. - Exact-head CI and security checks pass on `f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero unresolved review threads. Optional Paperclip Review completed neutral with a 5/5 static assessment of all 11 changed files and no findings; it did not run tests or hosted acceptance. All 57 current-head checks are terminal: 54 success, two skipped and the optional neutral review, with no failures or pending checks. - The real hosted workflow confirmed that the deployed absence repair recorded containment. A later ordinary UI Retry was accepted and saved a durable request, but it remains behind the historical source hold. The useful existing review remains unchanged. This follow-up is not deployed; a live successful successor and final task disposition remain unverified. No missing files are claimed recovered. ## Risks - The existing task reconciliation sweep can restage a deferred chat action. This reuses its receipt and generation; it does not grant new execution or publication authority. - Dispatch must keep the board Retry identity separate from the connected request's principal. The regression checks these bindings and preserves current chat authorization at each phase. - Historical holds reuse a strategy only from the locked same-run recovery that names the current Retry intent. A different run, malformed recovery record or independent source hold remains fenced. - Unknown executors without a matching active recovery still fail admission. Promotion and execution require containment even when admission saved a wait. - No schema or API shape changes. No credential changes, manual deployment or service restart. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:51:34 -07:00
DottaandPaperclip	898144f4b3	fix: stop idle execution-control recovery polling (#17094) ... Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-21 09:54:28 -05:00
DottaandPaperclip	d33b40a5c3	fix: recover saved work and provide reliable task Retry (#16965) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 19:58:53 -05:00
DottaandPaperclip	bc091f0a21	fix(recovery): recover retained workspaces before saved messages (#16906) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stopped run can leave useful files in its original sandbox. > - The server keeps that sandbox when it cannot confirm file copyback. > - The task can then save new user input but offers no safe way to recover the files. > - A retry of the old agent turn can repeat work that already finished. > - This pull request recovers files through the existing workspace owner and delivers only saved human input. > - The user can recover a task without certifying infrastructure facts or losing completed work. ## Linked Issues or Issue Description Refs #16883 and #16835 for publication receipts and verified process termination. Related: #16478 clears stale holds after copyback already succeeded. This change performs the missing export and records its proof. #16901 manages the main GitHub response comment and does not perform workspace repair. **What happened?** A failed run retained its workspace. A later user message stayed saved behind a file-repair hold. The task asked the user to verify process termination and file recovery but offered no repair control. Sending a message could show an optimistic Working state and move the task to Todo before admission succeeded. **Expected behavior** The harness should recover the original files when it can prove ownership and termination. It should preserve newer edits and completed operations. Only waiting user input may start another turn. Missing evidence must keep the source held with a specific explanation. **Steps to reproduce** 1. Interrupt a sandbox run before workspace finalization succeeds. 2. Retain its exact allocation and save a user follow-up. 3. Observe the file-repair hold and missing end-user repair path. ## What Changed - Add source-bound background recovery for native and legacy runs. An accepted native result is not required. - Share the physical copyback lock and durable owner. Resume only the exact stopped allocation, export files, verify persistence and obtain a fresh stop receipt. - Persist retry limits and idempotent requests on the existing recovery record. Limit each server to two background exports. Add an indexed candidate scan and a separate cleanup obligation. - Add a task-editor recovery endpoint. Deny agents and viewers. Reject caller-supplied stop or repair claims. - Deliver waiting human input once through ordinary admission. Preserve pause, approval, budget and assignment gates. Require historical conversational-adapter evidence for legacy-to-Runner continuation. - Include exact GitHub publication receipts in continuation context. Preserve unknown effects and the original failed run. - Show observed recovery progress, a transient-failure retry action and specific service blockers. Keep saved messages visible. Suppress false startup feedback and implicit Todo promotion behind a file hold. Hide unavailable workspace links. - Update execution semantics, capability maps and production-component stories. ## Verification - Red: the original read model failed the new recovery-progress assertion; the other 23 checks in that file passed. - Passed: recovery regressions with 28 cases with real PostgreSQL, files, tar and Git operations. Lifecycle provider calls are controlled test boundaries. Coverage includes native/legacy export, concurrent owners, rollback after export, interrupted archive export with a new database connection, lost replies, historical discovery, durable retry limits, separate cleanup and saved-input delivery. The actual candidate query uses its partial index with 3,000 unrelated history rows. A first automatic repair can bind verified old-controller stop proof without an accepted native result. Review also reproduced and fixed export-only continuation and cleanup lost after a committed repair. Cleanup remains discoverable without waiting input and after task reconciliation. - Passed: two GitHub integration cases verify that continuation context carries the existing publication receipt and does not write another response. Company and task boundaries are checked. - Passed: 293 continuation, workspace-sync, platform-receipt and OpenAPI checks. After file repair, actual dispatch retains pause, cancellation, approval, real budget-policy and assignment gates. - Passed: 174 route checks, including editor admission, agent/viewer denial and rejection of fabricated repair claims. - Passed: 351 task detail, workspace properties and comment-reopen checks after updating the new read boundaries and asynchronous workspace-link expectations. Two new route cases keep held tasks Blocked when a comment is saved. - Passed: 265 task-thread and recovery-notice checks. The broader continuation suite also passed; its new legacy migration case first exposed a gap that the focused recovery suite now covers. - Passed after rebase: `pnpm -r typecheck` and `pnpm build`. The later master merge adds an unrelated permission migration; recovery migration generation and safety checks passed after the merge. - Passed: token gates, module boundaries, map structure and migration checks. The recovery index is migration 0360 after current master. - Browser: inspected the production recovery component and its retry-to-queued transition with fixture data. The first dev dependency load failed; a reload after optimization resolved it. This is UI evidence, not a live provider recovery claim. - Every test shard passed on final commit `bbbc09efb4`. The older local `pnpm test:run` was stopped after the complete CI test matrix passed. Local full typecheck and production build passed. The first CI build compiled and booted, then lost its runner during the clean public npm-install check. An isolated retry passed the build, production boot and clean public npm-install verification. - Greptile reviewed final commit `bbbc09efb4` at 5/5. All review threads are resolved. - The change is not deployed. Hosted recovery remains unverified until the served revision includes this change. ## Risks - Recovery crosses process, provider and persistence boundaries. Missing or mismatched ownership, stop, descriptor or file evidence keeps the hold. It never selects a replacement allocation or silently starts over. - File recovery does not certify external action outcomes or authorize replay. New input still uses normal execution admission. - Historical sources without sufficient durable evidence can remain blocked. Destructive reconstruction stays separate. - The index migration is idempotent. It changes no task data. Snapshot pruning follows repository policy. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and browser tools. `gpt-6-astra` assisted earlier investigation. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 21:51:46 -05:00
DottaandPaperclip	7c7d2d8d2c	fix(runtime): continue from saved sources after workspace loss (#16690) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runtime must preserve saved work and prevent duplicate execution. > - Older cleanup could destroy a sandbox before its workspace export finished. > - The existing recovery controls require either the original sandbox or recovered original files. > - Surviving attachments can support a rebuild, but they do not prove that missing edits were recovered. > - This pull request adds an explicit reconstruction decision and one fresh continuation through the normal execution gates. > - Operators can continue useful work while the original failed run remains truthful. ## Linked Issues or Issue Description **What happened?** A historical failed export can have an accepted result, a prepared workspace descriptor, and a confirmed destroyed sandbox. Its notice can also say `new_source_execution_path` and disappear. The task then has neither a truthful original-file repair path nor a supported reconstruction path. **Expected behavior** Keep the missing-file obligation visible. Let an authorized operator choose surviving task attachments and record lost work and external action outcomes. Start one fresh continuation after current execution checks. Keep the old result and failed export intact. **Steps to reproduce** 1. Seed a failed native export with a prepared descriptor and an exact destroyed-lease receipt. 2. Resolve its notice with the historical `new_source_execution_path` bookkeeping value. 3. Read the task. The baseline hides the hold even though no source reconciliation exists. 4. With this change, choose **Continue from saved sources** and submit a reconstruction decision. Related public work: #16451 preserves healthy warm sessions. #16583 prevents destructive cleanup of unexported work. This change handles historical source loss that those fixes cannot undo. Searches of open and closed work found no equivalent reconstruction endpoint. ## What Changed - Add a board-only reconstruction endpoint with `runtime:manage` and task visibility checks. - Validate the exact run, result, descriptor and destroyed lease. Reject active copyback, competing owners, newer work, stale task status, and foreign attachments. - Record attachment hashes, missing work, and external action outcomes. Preserve the original failed run, accepted result, prepared descriptor and terminal failure. - Hold other continuation paths until the durable reconstruction intent admits its successor. Record that successor in the same transaction and preserve the retry count. - Add an operator control that distinguishes reconstruction from original-file recovery. Update the recovery runbook and capability maps. ## Verification - Exact baseline source returns no blocker for the destroyed-source fixture. The same fixture returns a required recovery hold with this change. - New real PostgreSQL and assembled API/heartbeat suite: 25/25 pass. It covers duplicate submissions, lost acknowledgement, context reset, source identity, task ownership, concurrent execution attachment scope, later human blocks, deleted sources and agent pause. - Existing export retry, explicit continuation and recovery-action suites: 386 cases pass. One new-fixture constraint error in the combined run was corrected; the full new suite then passed. - UI recovery suites: 46/46 pass across three files. They cover disabled actions, loss acknowledgement, failure retention and runtime authority. - `pnpm -r typecheck` and `pnpm build` pass. The first sandboxed typecheck attempt could not open the TypeScript IPC socket; the authorized rerun passed. - Module boundaries, map checks, token gates and diff whitespace checks pass. - OpenAPI route coverage: 21/21 pass. The first CI head exposed a missing endpoint registration; this head adds the registration and the focused suite passes. - Review identified a missing dispatch-time check and an isolated attachment cache. Both are fixed. Claim and final handoff recheck the exact successor, sources and current execution gates. A controlled post-claim operator block prevents the assembled heartbeat from entering the provider. The shared attachment cache refreshes uploads and removes deleted selections. - Final focused run: 180/180 pass across six suites, including reconstruction, existing dispatch/stale-queue contracts and UI recovery. - A second review reproduced cancellation by an already satisfied historical connection card while the task was `todo`. Read-only connection availability now works before checkout; active-task mutation checks remain strict. The baseline failed two cases; after the fix all 74 connection and reconstruction tests pass, including revoked access, pending questions and pending approvals. - All current-head CI gates pass on `9d197236da11fa24f1ceadf5589cb2b81179d824`, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 for that exact head with no unresolved threads. The full local `pnpm test:run` is still running its serial server group; no full local pass is claimed. Live staging recovery is pending deployment. ## Risks - Reconstruction cannot restore missing bytes. The operator must identify surviving sources and actions that must not be repeated. The agent must verify those attachments before rebuilding. - This path is limited to exact, confirmed destroyed remote sources with an accepted result. Retained or uncertain sources keep their existing repair path. - No schema migration, credential change, budget increase or warm-session policy change is included. Normal pause, ownership, budget, approval and dispatch gates still apply. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-17 22:57:10 -05:00
DottaandPaperclip	00d944f96c	fix(chat-channels): resume follow-ups after Slack Stop (#15737) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Chat channels let verified users stop an agent and send another message from Slack. > - Native execution needs the stopping user's identity to recognize a deliberate Stop. > - Slack omitted that identity, so a stopped run could create a recovery hold. > - A pending AI login request for another user could then block the new message. > - This pull request records the verified stop actor and scopes AI login waits to the current user. > - Users can start a fresh turn after a verified Stop while access and approval gates remain in force. ## Linked Issues or Issue Description **What happened?** A user stops a native run from its bound Slack thread. Slack confirms the Stop. The user sends a follow-up and receives a queued acknowledgement, but execution does not start. The run can acquire a recovery hold. A pending AI login request addressed to another user then blocks explicit continuation. **Expected behavior** An acknowledged operator Stop ends the current response. A new authorized user message starts through normal admission after the previous execution has stopped. An AI login request for another user stays pending but does not gate this user's turn. Tool access, approvals, unknown identities, and unconfirmed cleanup still block execution. **Steps to reproduce** 1. Start a native agent run in a Slack thread with a pending AI login request addressed to a different user. 2. Stop the run with the signed Slack Stop action. 3. Send a new message from a verified linked user. 4. Observe the recovery hold and the pending-decision message on the saved follow-up. Refs: #15421 for the related verified linked-user authority path. That change separates guest and trusted tasks; this change handles cancellation and login wait scope. ## What Changed - Pass the verified Slack user's identity to canonical run cancellation. - Scope AI login waits by the current run's responsible user, with a company-scoped lookup. - Use the same wait predicate for explicit continuation, native execution, and recovery decisions. - Admit already-saved native follow-ups posted after Stop through the existing continuation checks. Keep pre-stop and delivered messages held. - Keep pending cards and their authorization state intact. - Add regression tests for signed Slack Stop, fresh-turn admission, and same-user, unknown-user, tool-access, and company-boundary gates. - Give the two affected database setup hooks the standard 90-second budget. - Document Slack Stop and user-scoped AI login waits. ## Verification - Red: signed Slack Stop omitted the cancelling user; direct continuation and already-saved post-Stop continuation both returned `null`. - Green: 57 continuation/audience checks passed, including legacy operator Stops, pre-stop input, delivered input, same-user login, unknown identity, company boundaries, cleanup, ownership, pauses, and approvals. - The combined signed Slack Stop and original gate check passed: 39 tests. - Existing native question/completion suite: 33 tests passed. - `pnpm -r typecheck` passed. - `pnpm build` passed. - Rebased regression check: 18 tests passed on current master. The reviewed database-support setup change passed all 7 audience checks. - Full latest-head CI passed, including all server, chat, package, serialized-server, Runner, browser, build, and canary gates. - The duplicate local `pnpm test:run` was stopped after the full CI matrix passed. It was slow on the contended host; no failure had been reported. The focused reproduction command is: ```sh pnpm exec vitest run server/src/services/issue-question-context.test.ts server/src/services/explicit-native-continuation.test.ts server/src/__tests__/chat-channels.integration.test.ts --fsModuleCache --testTimeout 60000 -t 'AI connection wait audience|scopes connection waits|resumes an already-saved follow-up|durably retries a signed Slack session stop' ``` - Manual evidence: inspected the bound thread's cancelled run, queued messages, recovery hold, and login card. The staging deployment has not been changed. ## Risks - AI login wait behavior changes only when both the addressee and current responsible user are known and differ. Missing identity retains the gate. - Starting a new turn does not grant access or answer the old card. Credential selection still uses the new run's authority. - Process, remote cleanup, recovery, task ownership, pause, budget, and approval checks remain in place. - No schema migration or public API change. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, browser inspection, and code execution. The exact runtime model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 16:47:21 -05:00
Devin FoleyandPaperclip	f1c44b7b56	Preserve remote work when required workspace restore fails (#15479) ... Persist exact source-retention obligations before run finalization and protect them across cancellation, restart and task changes. Require board-authorized repair evidence without replaying old work or changing current task ownership, state or locks. Hide an unavailable retry action and document operator recovery and retention costs. Validated with 674 scoped regressions, 224 combined integration tests, full typecheck/build, independent safety reviews and green CI with Greptile 5/5. No historical file recovery is claimed. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-07 13:29:44 -07:00
Devin FoleyandPaperclip	2e67ea8dc7	fix: renew explicit continuation authorization for bounded retries (#14987) ... Preserve exact user-authorized continuation receipts across bounded retries and retain the task claim through owned retry admission. Stop, reassignment, superseding input, and active cleanup continue to block unsafe continuation. Validation: exact-head Greptile 5/5, passing CI, no unresolved review threads, and a clean merge. Detailed verification and limitations are recorded in the pull request. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-05 10:57:57 -07:00
DottaandPaperclip	94f6f3eb47	fix: recover historical interrupted native preparation (#15020) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task conversation must accept new user input after an interrupted startup. > - Native startup begins with a legacy preparation row before runtime selection. > - Older builds did not retain the startup cancellation receipt on that row. > - The immutable adapter claim and expired controller can still prove that no provider started. > - This pull request uses that narrow proof for explicit Retry and saved user input. > - The benefit is a conversation that recovers after an upgrade without repeating old work. ## Linked Issues or Issue Description Refs #15015. Related #13293 covers retained process evidence after provider startup. This change covers interrupted preparation before native runtime selection. **What happened?** After an upgrade, a task stopped during native preparation can still show automatic recovery stopped. A new user message saves but does not start. Retry is absent because the historical run has no cancellation receipt. **Expected behavior** Offer Retry and admit new user input when immutable run evidence proves that no provider started and cleanup is complete. Preserve incomplete or contradictory evidence as a recovery hold. **Steps to reproduce** 1. Retain a cancelled run with the Paperclip Runner adapter claim, an unresolved runtime, and the preparing stage. 2. Keep its old controller boot ID and expired lease. Retain no native identity, coordinator, result, process identity, or provider events. 3. Upgrade from a build that did not save the startup cancellation receipt. 4. Send a new user message or select Retry. Confirm that one fresh turn starts. 5. Repeat with an active controller, a provider launch, or unfinished cleanup. Confirm that execution stays held. **Paperclip version or commit** Reproduced on `cc67d4e1d` with a historical interrupted preparation row. **Deployment mode** Authenticated private self-hosted server, built from source. ## What Changed - Recognize historical interrupted native preparation from immutable run evidence and an expired controller from another server boot. - Reject adapter invocation evidence in the startup proof. Keep process and environment cleanup checks. - Apply the same proof to saved user messages in the bounded recovery worker. - Recheck saved-message eligibility under the existing task and run locks. Keep normal ownership, decision, pause, and budget gates. - Add regression tests for Retry, a new message, concurrent saved-message recovery, and contradictory evidence. Document the compatibility rule. ## Verification - Red: Retry, new-message recovery, and saved-message recovery fail on the parent commit. - Green: 185 continuation tests, 335 process-recovery tests, and 78 queued-comment route tests pass. Two additional red regressions cover partially delivered saved queues and pass after the admission fix. - Workspace `pnpm -r typecheck` and `pnpm build` pass. - Final head `48db53f4f`: all 54 CI checks pass; two optional Storybook checks skip. Greptile is 5/5 with zero unresolved threads. - The local monolithic `pnpm test:run` was stopped after CI passed. One unrelated workspace case failed in that long run; all three workspace reconciliation cases pass in isolation. No complete local monolithic pass is claimed. - After merge, deploy the exact merged commit and verify recovery through the normal task composer. ## Risks Historical compatibility could grant a new turn without enough startup evidence. The proof requires an immutable native adapter claim, an unresolved preparing stage, no result or native identity, an expired controller from another server boot, and no invocation or provider evidence. Environment cleanup remains mandatory. The fix does not replay old input or change historical run results. No schema or dependency change. ## Model Used OpenAI Codex, an agent based on GPT-6. The exact runtime model ID and context window are not exposed in this session. Used reasoning, repository tools, code execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-02 23:08:05 -05:00
DottaandPaperclip	cc67d4e1d8	fix: preserve steering and recover stopped task conversations (#15015) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task conversation must let a user guide a running agent and resume stopped work. > - The active run owns its input protocol, even when the user changes the next model or effort. > - Queue delivery waits for a provider receipt, which must be able to persist during the request. > - A stopped startup also needs a clear user action that passes normal task admission. > - This pull request fixes steering delivery, makes queue actions immediate, and restores explicit continuation. > - The benefit is a responsive conversation that can recover without losing saved input. ## Linked Issues or Issue Description **What happened?** A queued message could change from Steer to Interrupt while a native run prepared. A steer request could wait on its own database lock and fail to deliver. A stopped startup could then leave the conversation without a working Retry or message continuation. Interrupt also waited for the server and showed a toast. **Expected behavior** The active run keeps its input protocol. Steer delivers input to that run. Steer and Interrupt clear the submitted queue rows and show the input in the conversation immediately. Failed delivery restores the latest queue with an inline error. An eligible stopped run offers Retry, and authenticated user input can start a fresh turn through normal task admission. **Steps to reproduce** 1. Start a task with a native Paperclip Runner. 2. Change the selected model or effort while that run prepares. 3. Queue a message and press Steer. 4. Observe the provider receipt and queue state during the request. 5. Stop a startup before its provider process begins, then try Retry or send a new message. 6. Repeat queued delivery with a legacy runner and press Interrupt. **Paperclip version or commit** Reproduced on the parent of this branch, `59c07ede7`. **Deployment mode** Authenticated private deployment. The fixes also cover local task conversations. Related work: Refs #12834, Refs #13354, Refs #13275. The open refactor in #13160 moves the same queue route; it does not fix the receipt lock or stopped-run continuation addressed here. ## What Changed - Select queue behavior from the active run's immutable dispatch and runtime resolution. - Leave the run row unlocked during provider acknowledgement, then lock and read it before merging the receipt. - Retain queued input if the target run stops during that wait. Keep inline delivery errors visible after empty queue updates. - Permit exact Retry and authenticated continuation after verified native startup cancellation. Preserve pause, approval, budget, ownership, and process-stop gates. - Carry undelivered native queue input into a fresh turn once the old execution is confirmed stopped. - Show Steer and Interrupt input in the conversation and clear submitted composer rows immediately. Restore the latest queue inline on failure. Remove delivery toasts. - Keep optimistic delivery stable across stale polls, empty queues, and paginated history. Preserve classic Interrupt error handling. - Document recovery and optimistic delivery behavior. Add regression tests across server, shared queue projection, and UI boundaries. ## Verification - Red-green regression tests reproduced the queue protocol, receipt lock, stopped-startup continuation, and optimistic delivery failures. - The focused server route, continuation, queue, and runner boundary suites passed during implementation. - The queue-route suite passes with 78 tests. The three complete conversation UI suites pass with 347 tests. - UI typecheck, production build, and `pnpm check:token-gates` pass. - Workspace `pnpm -r typecheck` and `pnpm build` pass. The local monolithic `pnpm test:run` is still running; remote CI verifies the complete suite on the latest commit. - All CI gates pass on `bd9031ad56abfcde13d13a13488c1b9217c2fd3a`, including the full test shards, runner verification, browser E2E, typecheck, release registry, and canary dry run. - Greptile reports 5/5 for that commit. Both review threads are resolved. ## Risks This changes queue display and explicit continuation admission. The UI must restore rejected delivery without losing other-session edits. The server must preserve concurrent provider result updates and must not resume a process whose stop is uncertain. Focused tests cover these boundaries. This change has no database migration. ## Model Used OpenAI Codex, an agent based on GPT-6. The exact runtime model ID and context window are not exposed in this session. Used reasoning, repository tools, code execution, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-02 22:07:14 -05:00
DottaandPaperclip	7a52dcdc74	fix: repair MCP validation and cancelled execution recovery (#14951) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The tool gateway gives agents access to connected services. Recovery controls what happens when a run stops. > - Generated tool names can exceed the provider limit after the MCP client adds its prefix. > - The same invalid definition can fail each automatic retry. A cancelled run can also hold saved messages without showing its cause. > - This pull request bounds tool names, stops configuration retries, and retains cancellation evidence. > - It shows the stopped run and admits saved input only after the existing safety checks pass. > - The benefit is a clear recovery path that preserves operator Stop and prevents duplicate message delivery. ## Linked Issues or Issue Description **What happened?** A long connected MCP tool name makes the provider reject the entire request. Automatic recovery repeats the invalid request. Separately, unexpected legacy cancellations can leave saved input behind a recovery hold. The notice does not identify the stopped run or its cause. **Expected behavior** Complete MCP names fit the provider limit. Tool-definition errors require configuration repair. Cancelled runs retain their source and reason. The recovery notice shows the cause and saved-message count. Verified unexpected cancellations can start a fresh turn through the existing admission checks. **Steps to reproduce** 1. Assign an App gallery connection with a long application key and tool name to a Claude agent. 2. Start a run. The provider rejects a name over 128 characters, including its MCP prefix. 3. For cancellation recovery, stop a legacy provider turn without an operator Stop request and send a user message while the recovery hold is active. 4. Inspect the recovery notice and the deferred message queue. **Paperclip version or commit** Rebased onto master at `cf8ad63c806685bfd7c48e3ed4a919d61a7c55f1`. **Deployment mode** Hosted or self-hosted server with legacy Claude or Codex execution. Related public work: - Refs #14017. That PR caps name segments. This PR preserves existing short names and uses stable hash aliases for long complete names. It also covers classification and recovery. - Refs #4510. That PR adds a cancellation-source column. This PR records bounded evidence in the existing run result, without a migration. - Refs #12552 and #4506. Those PRs suppress recovery after operator cancellation. This PR preserves operator intent and uses the existing continuation gates. ## What Changed - Bound gateway names with the full provider prefix in the 128-character budget. Retain the original upstream tool name for dispatch and permissions. - Classify invalid tool definitions as configuration failures before diagnostic redaction. Stop automatic retries and continuation attempts for that error code. - Persist cancellation source, expectedness, initiator, reason, and time. Preserve recorded Stop intent when adapter results arrive. Report unexpected started cancellations with closed diagnostic labels. - Show the run cause, saved-message count, and Inspect run link. Offer Continue for eligible unexpected cancellations. Require verified provider stop, empty tool inventory, ownership, and the existing pause, budget, approval, and dependency gates. Use the existing queue for single delivery. - Add regression coverage and update the execution, MCP gateway, and run-log documentation. ## Verification - `pnpm -r typecheck` and `pnpm build` passed. - `pnpm check:token-gates` passed. - Ran `pnpm test:run` and completed its workspace and serialized groups. Initial resource and timing failures passed on isolated reruns. All 149 serialized route suites passed. - Reran the changed server, adapter, and UI suites after the rebase. Coverage includes long-name upstream dispatch, configuration retry suppression, cancellation evidence retention, privacy labels, oversized run projection, and concurrent saved-message delivery. - `pnpm test:e2e tests/e2e/legacy-failure-continuation.spec.ts` passed all six browser scenarios. The recovery notice shows the run cause and inspection link, and each recovery entry point reaches one new response. - Added database-backed checks for active, removed, paused, unavailable, and disabled chat connections. The final continuation and recovery-notice suites passed 167 tests. Externally bound chats hide board Continue and show a usable next action. - All 55 GitHub checks passed on `42afbf1371dcaeb72646e3d8f65c19ff7cddf8de`. Two unrelated Storybook jobs were skipped by their normal conditions. Greptile reviewed that commit at 5/5 with no findings and no open review threads. ## Risks - Long tool names change to aliases. Existing short names stay compatible. The original connection and upstream name remain the dispatch authority. - Invalid tool definitions no longer get automatic retries. An operator must repair the configuration before a new attempt. - Continuation changes apply only to positively identified unexpected legacy cancellations with complete empty tool inventory. Operator Stop, unknown historical cancellations, outstanding tools, and unverified provider termination keep their holds. - No database migration. The added projection fields are optional. Cancellation reason and initiator IDs remain local run evidence; Sentry receives only closed source and initiator-type labels and expectedness. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository editing, shell execution, and GitHub tool use. The runtime does not expose the exact model variant or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-02 13:47:59 -05:00
DottaandPaperclip	8ec4b84e1c	fix(chat): resume messages after failed runs without duplicate delivery (#14857) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A user can send a new message after a native run fails. > - The server checks that the old execution has stopped before it starts a fresh turn. > - A failed run can retain a result accepted before checkpoint or cleanup failed. > - The continuation gate treated that saved result as active recovery and held the new message forever. > - This pull request removes that false liveness signal while retaining controller, process, environment, and authorization checks. > - Live staging then exposed a second defect: chat admission created a successor without consuming the original deferred receipt, so completion delivered the message again. > - Consume that exact receipt atomically with admission, while preserving separate turns for later chat messages. ## Linked Issues or Issue Description **What happened?** A new user message stayed in the queue with `controller_settling` after the previous run had reached `terminal_failure`. The old coordinator had no lease owner but still had a `resultId`. Its remote environment had a verified stop receipt. **Expected behavior** Start one fresh turn after execution has stopped and normal admission checks pass. Preserve the failed run and its accepted result as history. **Steps to reproduce** 1. Accept a native result, then fail checkpoint or cleanup and exhaust recovery. 2. Retain the result ID on the terminal failure record and stop the execution environment. 3. Send a new user message. Before this fix, it waits forever for the finished controller. **Paperclip version or commit** Reproduced in a database-backed regression test on `26900655b`. **Deployment mode** Server with a native runner and remote sandbox. Local process stop checks also apply. Related: https://github.com/paperclipai/paperclip/pull/14775. Searched existing PRs for retained-result continuation fixes; no duplicate found. ## What Changed - Remove the retained-result veto for terminal failures. - Keep controller ownership, successor, process, environment cleanup, pending decision, and ordinary admission checks. - Add regressions for retained results, active execution, missing stop evidence, and delayed remote cleanup. - Atomically consume the resumed receipt in agent chat, even though chat does not coalesce other queued messages. - Reproduce completion-time duplicate promotion, race cleanup against periodic recovery, and prove a subsequent chat message keeps its own turn. - Document that a saved result does not make a terminal failure active. - Keep exhausted workspace export on its separate repair path, tested through the production finalizer. ## Verification - Red: retained-result admission failed with `controller_settling` before the original fix. The new chat-specific regression then reproduced duplicate promotion when the first reply finished. - Green: 406 tests across native continuation, workspace-export recovery, and the wake-queue module passed on `cbc531cc0`. - The chat regressions exercise real Postgres transactions, simultaneous recovery callbacks, successful completion, the production queue-drain use case, and repeated drain attempts. A distinct follow-up remains a separate turn. - `pnpm -r typecheck` and `pnpm build` passed on `cbc531cc0`. - The earlier full local test run encountered a timeout and follow-on failure in unchanged AI connection-adoption tests; all 50 tests passed on isolated rerun. That local run was stopped after the full CI test matrix passed on the earlier head. - All 54 CI checks passed on `cbc531cc0` (2 skipped), including the full test matrix and browser shards. One unchanged interaction-route test returned HTTP 500 on its first CI attempt; its full 84-test file passed locally, and the failed shard passed on one targeted rerun. - Greptile reviewed `cbc531cc0`: 5/5, no unresolved findings. - Live staging first verified that the original saved message resumes and receives a successful response; that test exposed the duplicate now covered above. - Deployed exact commit `cbc531cc0410e1ef6e8811c6c5c014c3528351ed` to the affected staging workspace; deployment verification, health, authentication, and startup recovery passed. - Submitted a fresh message through the browser. The agent replied in 39 seconds; server records show exactly one successful run, native phase `committed`, no error, and an empty queue. A later check more than a minute after completion found no duplicate run. ## Risks The change affects admission after native execution failure and consumption of a resumed deferred receipt. A fresh turn must never overlap the prior execution, and consuming one chat receipt must not absorb later messages. Tests retain the controller, process, and remote-stop guards. This change does not migrate data, apply an old result, or reset the old retry budget. ## Model Used OpenAI Codex (GPT-6). The exact runtime model identifier and context window are not exposed in this session. Used reasoning, repository inspection, code execution, database-backed tests, and browser inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-01 14:43:02 -05:00
Devin FoleyandPaperclip	bd6caf51bb	fix: preserve restore failure results and stop unsafe retries (#14035) ... Preserve agent output and earlier execution errors when workspace restore fails. Report the restore phase and confirmed saved-plan links. Require verified repair before retrying unsafe archives, while preserving approval states and the retry budget. Verified with full CI, 506 focused regression tests, and Greptile 5/5 with all review threads resolved. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-25 11:34:31 -07:00
DottaandPaperclip	74a9730acb	fix: continue native agent chats after worker loss (#13813) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat uses native workers to run Claude and Codex conversations. > - A worker crash leaves a cleanup hold because its provider did not acknowledge suspension. > - A new user message must not reuse that unverified session or repeat old tool calls. > - The existing continuation path can preserve history and start a fresh session, but local cleanup ownership remained held. > - This pull request verifies the stopped local owners and releases only their cleanup hold for a new user turn. ## Linked Issues or Issue Description Refs #13775. **What happened?** After a native worker crashed, both providers retained cleanup quarantine. A saved plan survived, but the conversation could not produce another answer. **Expected behavior** Once the old worker and provider process groups have stopped, a new user message can continue in a fresh session with the saved work and prior action history. **Steps to reproduce** Run the opt-in `agent-chat-qualification` suite with case `worker-crash-retry` on native Codex and native Claude. The fixture saves a plan, kills the exact worker through a Linux pidfd, releases a local read-only brief, and sends a new message. ## What Changed - Verify the exact local worker stop receipt, provider identity receipts, released leases, and retained state before retiring a native cleanup hold. - Recheck process liveness and state before admission. Keep the old run and durable session files intact. - Use the existing explicit conversation continuation path. Generic Retry remains blocked for cleanup quarantine, including on the old failed-run marker after a successful continuation. - Extend the live oracle to require a successful fresh session, correct predecessor context, unchanged plan, one original message, and one answer containing a reference introduced after the crash. - Add physical-proof and database-backed admission tests. Document the precise qualification scope. ## Verification - Live Product E2E: **2/2 passed**, **2/2 cleanup passed**, with real native `gpt-5.6-sol` and `claude-sonnet-5`, Chromium, server, database, and public APIs. - Core recovery proof source: `3592b04c2bc76e23795fcdf964720e38a409dc4d`. Suite definition version 8: `9867367994d81a0c726956d91f2c7fddab6417a12f41b5cef3f7e62f3be417da`. - [Core recovery campaign](https://github.com/paperclipai/paperclip/actions/runs/35741746990) · [Public evidence report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35741746990-1/). - Both cells verify the real crash boundary, blocked generic Retry, unchanged saved plan, one original prompt, one fresh successor with predecessor context, and one run-attributed answer containing the post-crash reference. Billing coverage is partial because the crashed runs did not report complete usage; missing cost is not zero cost. - [Master baseline](https://github.com/paperclipai/paperclip/actions/runs/35737364443): both providers stopped at quarantine, with cleanup passing. - [Follow-up baseline without the fix](https://github.com/paperclipai/paperclip/actions/runs/35738638866): Codex produced a complete red result. Claude reached the same error, but its artifact upload was canceled. - [Complete Claude baseline with the same version-8 definition](https://github.com/paperclipai/paperclip/actions/runs/35740555177): red at cleanup quarantine, cleanup passed, source `6479a90c5754044356b39a9278b9a3e92ce8e55e`. - Earlier candidate attempts remain retained: [first](https://github.com/paperclipai/paperclip/actions/runs/35738449214) passed Codex and found a Claude fixture wait race; [second](https://github.com/paperclipai/paperclip/actions/runs/35740408065) exposed the normalized session-open receipt mismatch. Both corrections are in the final source. - Targeted server suites: 570 passed before the final two additional receipt regression cases. The physical-proof suite, including those cases, passed 46/46. Eval oracle and catalog: 40 passed. Server and Product E2E typechecks passed. - **All 54 PR checks passed on final head `db6f775df`**, including repository typecheck, build, tests, browser shards, and canary dry run. The canary job required one retry after its runner received a shutdown signal. On the earlier core proof head, two timing-sensitive tests passed in isolation and on a single CI retry. - Final UI regression checks: 5 passed; UI typecheck and token gates passed. Updated eval oracle/catalog: 40 passed; eval typecheck passed. - Final version-9 two-provider campaign: **2/2 passed, 2/2 cleanup passed**, including the browser assertion that the quarantined historical run never regains Try again. [Final campaign](https://github.com/paperclipai/paperclip/actions/runs/35747416013), source `db6f775dfff405e1514ec02fedb0450d42c7dad2`, definition hash `bf5abf1cc45cb6dad4e082fbf818b8fa0f4d8c282776a7e98762b22919eadab7`. [Final public evidence report](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-35747416013-1/). Final screenshots and retained state inspected for both providers; cost coverage remains partial. ## Risks - Missing or conflicting stop evidence keeps the conversation blocked. This change does not kill an unverified process. - This qualifies new user input after local worker loss. It does not enable automatic replay, exact-session recovery, remote crash recovery, or native onboarding defaults. - Old action outcomes remain part of the continuation. A process exit is not proof that an action did not happen. - No schema migration or production prompt change. ## Model Used OpenAI Codex, GPT-6. The runtime does not expose a more specific model identifier or context-window size. Used reasoning, repository inspection, code editing, shell tools, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-22 13:01:45 -05:00
DottaandPaperclip	9f30eb10dd	fix: reduce chat latency and preserve managed session reuse (#13710) ... ## Thinking Path > - Paperclip manages AI agents and keeps their work attached to tasks. > - Chat connectors carry user messages and agent replies between a provider and those tasks. > - Each extra startup and context reset delays a reply. > - Managed account metadata was lost during adapter decoding, so compatible follow-ups started fresh. > - This branch fixes the reset and measures the remaining preparation, execution, and delivery costs. > - The changes must preserve account isolation, authorization, durable output, and recovery ownership. ## Linked Issues or Issue Description Refs #13699. The related service lifecycle work in #13410 and #13408 is separate; this branch focuses on task-bound chat response latency. **What happened?** Managed AI follow-ups started new provider sessions even after their configuration fingerprint stayed stable. The Codex codec removes unknown fields. The resume check then read the removed credential identity and treated it as a credential change. **Expected behavior** Compatible follow-ups resume the correct provider session. Changes to credentials, responsible users, permissions, or task configuration retain their reset behavior. **Steps to reproduce** 1. Use a Slack connector with a managed AI connection. 2. Send a message, then send a same-thread follow-up. 3. Inspect the configuration reset reason and the provider session identity. **Paperclip version or commit** Reproduced on `2a99de80ec52db01eead901f28323926ceaf3c1d`. **Deployment mode** Cloud staging with a native Codex runner. ## What Changed - Read saved credential identity before adapter decoding discards it. - Remove the internal credential identity from adapter-facing session params. - Test the real Codex codec and missing, changed, or unmanaged identity cases. - Preserve configured warm Codex runners and flush refreshed credentials after every turn. - Fence detached or closing session handles from successor credential ownership. - Stage current Codex launch credentials after restoring durable session history, uploading launch assets only once. - Reuse a runner binary already in the retained sandbox only when its SHA-256 matches the controller-owned artifact; still verify required capabilities before launch. - Lock the task before the run when saving results, preventing deadlocks with task updates. - Scope reusable projectless sandboxes to the company, environment, task, agent, and runtime configuration; verify Daytona sentinels for that scope. - Admit a new authorized chat message after a fully committed failed run and verified process cleanup. - Send compact deltas for verified plain-text Slack continuations. Match the actual prior run and current comment identity/body; exclude edited historical comments and prior agent output, preserve genuine brief edits and the full bootstrap fallback. - Keep attachments, omitted input, questions, approvals, recovery, and other providers on their existing framing. - Document managed session compatibility, credential lifecycle, and compact continuation boundaries. ## Verification - Workspace/session coverage: 156 tests passed. - Native session and credential ownership coverage: 390 tests passed, including exact artifact reuse, mismatches, failed probes, timeouts, and explicit artifact overrides. - Explicit continuation and durable chat authorization coverage: 172 tests passed. - Session resume and launch preparation coverage: 416 tests passed. - Result persistence coverage: 15 tests passed. The new concurrency test reproduced a PostgreSQL deadlock before the lock-order fix. - Environment lifecycle coverage: 92 tests passed, including projectless reuse and task/agent isolation at both selection and atomic handoff. - Daytona plugin coverage: 237 tests passed; 6 gated tests skipped. Standalone plugin build passed. - Compact Slack continuation and native resume coverage: 69 tests passed, including full-bootstrap retention, matching message authors/bodies, current-delivery selection, rejection of duplicate identities and historical comments, brief edits, and attachment/recovery fallbacks. - Final frozen-head `pnpm test:run` on repository-supported Node 26: 668 suites passed, 3 skipped, 1 failed; 12,797 tests passed and 82 skipped. The sole failure was a local `socket hang up` in `issue-recovery-actions.test.ts`, not an authorization assertion mismatch. All 57 tests in that suite passed three fresh reruns, and the suite passed latest-head CI. The full local invocation is therefore not claimed green. - An earlier Node 24 full run exposed an unrelated macOS symlink-cleanup failure; that 11-test catalog suite passes on Node 26 and in CI. No test behavior or timeout was relaxed. - Full local typecheck and build passed. Latest-head CI is green; Greptile is 5/5 with no unresolved review threads. - Two real Slack baseline replies took 25.1 and 24.6 seconds (24.9-second mean). Three same-thread signed probes on this head took 23.8, 23.9, and 22.7 seconds (23.5-second mean). This is a small sample and a modest wall-clock improvement, not a large or statistically established speedup. - In that same thread, uncached provider input fell from 8,514 tokens before compact input to 694–765 tokens afterward. The current delivery uses a 362-character delta; the full 19–21k-character bootstrap remains available for failed resume. Verified runner artifact preparation fell from about 1.2 seconds to 0.6 seconds. - A fresh thread created a separate task, sandbox, and provider session with full bootstrap (24.4 seconds). Its follow-up reused its own sandbox/session and compact input (28.3 seconds, including 16 seconds of model execution). Model variability and process startup remain substantial. - A signed duplicate webhook produced exactly one user comment, one successful run, and one final Slack reply. Slack's API independently confirmed the actual replies and a public task URL without an internal or pool hostname. - Earlier signed probes verified recovery after a failed run and reuse across a server deployment. The final idle test observed Daytona report the sandbox as stopped, then delivered a new reply in 19.9 seconds using the same sandbox/provider-session identity and compact input. Slack’s API confirmed that reply. - Live probes use signed synthetic inbound webhooks and real outbound Slack delivery, read back through Slack’s API. The final browser recheck found the Mac locked and the Slack tab blocked by another extension, so this is not claimed as full UI E2E proof. - This is a review branch. Do not merge until the maintainer reviews it. ## Risks - Incorrect session reuse could mix account or task context. Missing or changed identities continue to reset, and existing authorization checks remain in place. - Warm mode remains opt-in. Remote warm mode requires a reusable sandbox lease. Retained processes keep credentials until they close, so idle expiry and ownership fences are required. - A fresh user message may continue after a committed provider failure. Approval, current authorization, process termination, and prior-result checks remain required. - Projectless sandbox reuse is task- and agent-scoped. Missing or mismatched ownership cannot replace an existing lease; existing workspace-scoped leases keep their scope. Opt-in reuse retains a sandbox per task/agent, so provider auto-stop and deletion policies still determine idle compute and storage costs. Fleet defaults are unchanged. - Compact prompts apply only after proven resume and a matching prior-run delta. Missing or specialized context falls back to full input; fresh sessions always receive the full bootstrap. - No schema or migration changes. ## Model Used OpenAI GPT-6 through Codex, with code editing, tool use, and test execution. The exact serving model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-20 13:12:44 -05:00
DottaandPaperclip	d0b67bfe71	feat: queue approvals and answers during active runs (#13539) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Users guide running agents through messages, questions, and approval cards. > - Messages already wait in a queue when an agent is running. > - Card responses did not appear in that queue. Some question answers also steered a later run without a user click. > - A fast approval could invalidate the agent's review handoff and cause it to stop its own run. > - This pull request gives card responses the same queue controls and preserves the exact response during delivery. > - Users can wait for completion or explicitly send the response with Interrupt or Steer. ## Linked Issues or Issue Description Refs #13517, which is merged. This PR targets master and adds queued interaction responses on top of the onboarding changes. Related continuation work: #10519 and #12866. **What happened?** Accepting a proposal while its source run was active left a saved response outside the message queue. The agent could then lose its review path, reassign the task, and cancel itself. Answers to older questions could also steer another active turn without a click. **Expected behavior** Save the response immediately. Queue its continuation behind the active run. Deliver it after completion, or when the user explicitly chooses Interrupt or Steer. Preserve approval revisions and answer choices. **Steps to reproduce** 1. Let an agent publish a confirmation card while its run is still active. 2. Accept the card before the agent finishes its review handoff. 3. Inspect the message queue and the task's next run. **Paperclip version or commit** Reproduced on da8a3876c with the onboarding changes from #13517. **Deployment mode** Local development from source. The fix covers legacy adapters and native Runner turns. ## What Changed - Project resolved cards into the existing queue as immutable responses. Keep answers and exact approval revisions. - Require an explicit click to steer a response into a compatible native turn. Use Interrupt when a fresh session is required. - Preserve typed response context through interruption, cleanup waits, and normal queue promotion. Keep the direct answer channel for a provider blocked on its original question request. - Accept the source run's review handoff after its card resolves. Reject stale agent reassignment that would orphan a queued response. - Add deterministic regression tests and an `accept-while-running` case to the first-task suite. Require recorded timestamp overlap before that case can pass. - Keep the first-task skill name out of user-facing messages. ## Verification - Red-green: the original route failed the queue regression; the changed route passes it. - Focused server/UI tests: 139 passed, including 64 queue-route tests. - Runner harness unit tests: 314 passed. - Server, UI, and Runner E2E typechecks passed. UI token gates passed. - Full repository typecheck and build passed. Server typecheck passed again after review fixes. - Review regressions: 165 queue/reopen route tests, 53 wake admission tests, and 18 run identity tests passed. Approval acknowledgement recovery and both message/approval arrival orders are covered. - Full local test run: 12,401 passed; three new admission regressions ran against a cached pre-fix module. A fresh run of that entire suite passed (53 tests). The complete CI suite passed on the final commit. - Previous-head CI at `c28e2ef12`: 32 checks passed and 2 optional Storybook checks skipped. Every server/workspace/browser shard, Runner verification, build, typecheck/release registry, canary, policy, and security check passed. Greptile: 5/5, no unresolved threads. Earlier interrupted CI workers were replaced by this fresh complete run. - After integrating the updated parent: 314 harness tests, 119 queue/admission tests, 44 onboarding/question-delivery tests, and 13 native recovery tests passed locally. Full repository typecheck and build passed. - Clarified the skill wording preference: routine replies describe the action without announcing the internal skill; direct questions and permission/security/execution disclosures remain truthful. - The paid `accept-while-running` scenario is registered for all four local first-task profiles. It has not been run against a model in this change. - Rebased onto the merged parent at `11921075a`; the resulting tree exactly matches the locally verified integration tree. Final-head CI on `b53054807` passed: 54 successful checks, 2 optional Storybook checks skipped, no failed checks. Every new server/browser shard, aggregate verify/e2e gate, Runner, typecheck, build, canary, and security check passed on the first attempt. Greptile reviewed this exact head at 5/5 with no unresolved threads. ## Risks - Responses now wait instead of implicitly steering another active turn. A provider blocked on the original question still receives its answer directly. - Approval receipts cannot be edited, discarded, or reordered as comments. This preserves the recorded decision. - Interruption must still prove that the prior execution stopped. The tests cover cleanup waits and duplicate delivery. - The new paid overlap case can be unexercised if the model finishes before the click lands. It cannot pass without evidence of overlap. - No database migration is required. This repairs the existing approvals and execution controls; it does not implement the roadmap's work-stream queues. ## Model Used OpenAI GPT-6 through Codex. The exact deployed model ID and context-window size were not exposed in this session. Capabilities used: agentic reasoning, repository inspection, code editing, terminal commands, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-16 14:44:36 -05:00
DottaandPaperclip	4cd7b40255	fix: recover new messages after historical native runs stop (#13405) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native recovery must distinguish a fresh user request from replay of failed work. > - Older runs can lose their process fields before a local stop receipt exists. > - A suspended durable session can still prove that the exact runner and provider session are idle. > - The message admission path ignored that evidence and kept new user messages blocked. > - This pull request uses the existing exact-state verifier for those historical runs. > - The user can start one fresh turn while the old history and unknown outcomes remain intact. ## Linked Issues or Issue Description **What happened?** A user sent a new message after a native run exhausted recovery. Paperclip saved the message but said the previous run had no verified stop record. The old runner was suspended, with no active provider turn or pending output. Its process fields had been cleared before stop receipts were added. **Expected behavior** A new user message starts a fresh turn when the exact retained session proves it is suspended and the other execution gates pass. **Steps to reproduce** 1. Retain a failed native run with a terminal controller, cleared process fields, and no process receipt events. 2. Retain its exact suspended runner state and idle provider state. Keep its recovery hold. 3. Send a new user comment. Before this fix, admission returns no successor. **Paperclip version or commit** Reproduced on master at d351e08de. **Deployment mode** Self-hosted server. The regression uses an embedded PostgreSQL test database and real durable state files. Related work: Refs #13270, Refs #13338. This adds compatibility for older stopped runs. Refs #13332 concerns separate recovery-hold scope rules. ## What Changed - Add exact suspended-state evidence to explicit native message admission for runs that predate process receipts. - Reuse the existing failed-retry verifier for run, runner, workspace, provider identity, and pending-work checks. - Reject this fallback if any server-authored process receipt or launch event exists. - Add a red/green regression with real message admission, duplicate delivery, dry-run behavior, and blocked-state cases. - Document the new-message recovery rule. ## Verification - Red: the historical suspended-state regression failed on master because admission returned null. Eight rejection cases passed. - Green: 469 tests passed across explicit native continuation and native session execution. - Full repository `pnpm -r typecheck` and `pnpm build` passed locally. - The complete Vitest CI matrix and all browser test shards passed. The duplicate local `pnpm test:run` was stopped after these CI results; it did not complete locally. - The first runner verification worker received an infrastructure shutdown signal during compilation. The single retry passed. - Greptile reviewed commit `6836d7310` at 5/5 with no findings. - Inspected the affected server's database and durable state read-only. It has the historical missing-PID shape and an exact suspended runner with no active provider turn, pending tools, or undelivered output. ## Risks - Incorrect idle evidence could allow overlapping work. The fallback requires an exact suspended root and rejects active or pending work, mismatched identities, missing files, and newer process evidence. - Normal task, controller, lease cleanup, decision, and active-run gates remain in force. - This does not resume old provider actions or reset recovery attempts. Unknown outcomes remain unknown. - No schema change or deployment is included. ## Model Used OpenAI Codex, GPT-6. The session does not expose the exact model snapshot or context-window size. Used reasoning, repository inspection, code editing, shell execution, and test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-14 09:45:34 -05:00
DottaandPaperclip	6cef9743c0	fix: deliver saved user messages after recovery stops (#13327) ... Deliver saved user messages after legacy recovery stops. Validate undelivered comments and queue ownership under the task lock, preserve the operator identity checks from #13315, and prevent duplicate successors. Add a recovery notice with Retry and inline errors, plus service, route, component, and browser coverage. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-12 18:08:04 -05:00
DottaandPaperclip	df984cbc2c	fix: dispatch queued legacy messages with operator identity and task permissions (#13315) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations save messages that arrive during an active turn. > - Legacy adapters deliver these messages in a later turn. > - A run can stop before the saved queue is delivered. > - The Interrupt button previously required an active run, so it could not release this queue. > - This pull request lets a board operator send the saved queue after the run stops and retries queues missed during finalization. > - Manual dispatch must use the clicking operator and must not require permission to create agents. > - The task can continue without a duplicate message or a second execution owner. ## Linked Issues or Issue Description **What happened?** A legacy task retained a queued message after its run stopped. Interrupt was disabled because the queue had no active target. Finalization and deferred message admission can also leave a queue without a successor. **Expected behavior** Interrupt sends the saved messages when no runner is active. Messages that arrive during normal completion are delivered automatically. An uncertain previous execution still requires proof that its process or sandbox stopped. **Steps to reproduce** 1. Queue a user message during a legacy conversation turn. 2. Let the turn stop or simulate a server restart before queue promotion. 3. Open the task with a deferred queue and no active run. 4. Try Interrupt. Before this change, the button is disabled. **Paperclip version or commit** Reproduced against `8f40b4ad4`. **Deployment mode** Legacy conversation adapter. The same persisted queue state is covered with an isolated PostgreSQL fixture. Related public work: #13275 adds active legacy interruption. #13291 addresses automatic sandbox conversation recovery. This change handles explicit saved-queue delivery and late queue promotion. ## What Changed - Accept a null Interrupt target while retaining queue identity, revision, company, and assignee checks. - Save the operator's request on the existing queue. Reuse normal admission after verified stop, including older messages, different authors, and queues whose original wake came from the system. - Strip interruption authority from caller-supplied wake payloads. Only the board queue route can persist that authority. - Retry durable interruption requests after restart and deferred queues after legacy cleanup. - Let an explicit Interrupt retry cleanup for its stopped run, including old ephemeral leases that recorded success without a provider stop receipt. Preserve retained resources, other lease owners, and the automatic retry limit. - Preserve the server's waiting explanation when normalizing and combining queue entries. - Revalidate the consumed board queue receipt at dispatch so a different message author does not cause setup failure. - Use the Interrupt user's execution identity for the new run. Preserve original message authors. Validate the receipt independently at startup and inherit the resulting identity on retry. - Persist authenticated board authority for ordinary manual wakes too. Adopting someone else's queued messages cannot switch a manual run to that author's permissions. Strip caller-supplied authority markers and retain private conversation ownership checks. - Keep the clicking user when a manual wake is merged into an older deferred receipt. Update its requester and payload in the same transaction. - Use the same current-queue/revision API on task details and pipeline conversations; show Interrupt after a legacy target stops. - Keep manual wakes out of active runs, including unscoped agent wakes. They receive their own execution identity; a matching receipt requester is not sufficient because an exact retry can retain a different originating identity. - Authorize both existing-agent wake endpoints with `agent:wake`, available to active non-viewer company members. Keep `agents:create` for hiring. Validate the stored task and current assignee before an exact task retry. - Reject viewer Interrupt requests before saving intent or stopping execution. Keep external chat retry authorization and per-action agent/user permission checks. - Preserve edits and discards until dispatch. Prevent another queue promotion when the same agent already has a successor. Keep independent reviewer recovery available. - Suppress cancelled/failed run toasts for intentional operator interruption. Keep ordinary runtime error notices. - Add UI, route, admission, restart, successor ownership, and toast regression tests. Document the behavior. - Reuse the existing socket reservation helper for both credential-quorum test cases after CI exposed an ambient-port collision. This changes test preparation only; production credential staging is still called exactly once. ## Verification - Failing regression tests reproduced the message-author identity bug and an operator's `agents:create` rejection before the fixes. - All 316 focused tests pass across eight route, queue, identity, authorization, continuation, and responsible-user suites, including the 44-test rerun of queue admission and actual startup after the final manual-wake restriction. Regressions reproduce cross-user merging both with and without a task, and same-requester receipt ambiguity. The cross-company existence guard also passes both tests. - Startup integration tests reach adapter execution under the clicking operator and retain that identity through follow-up. Coverage includes mixed authors, adopted queues, system-origin queues, restarts, forged or stale receipts, viewers, suspended memberships, changed assignees, private conversations, and caller-supplied authority markers. - The earlier queue/cleanup/UI regression suite passed 402 tests. The final review corrections pass another 180 tests across queue admission/persistence, real heartbeat startup, UI API, conversation rendering, and pipeline suites. Regression tests reproduced both review findings before correction. The final head has a 5/5 review with no unresolved threads. Full CI passes on `c2002979c`, including every general and serialized server shard, all browser shards, Paperclip Runner verification, typecheck, build, canary dry run, and the aggregate gates. - Full `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after the final application changes. CI identified an outdated task-page API mock after the shared helper extraction; the fixture now exercises the real helper, and all 131 task-page/API tests pass. The final application build passes with the additional manual-wake restriction. - CI exposed a pre-existing port collision in the Codex credential-quorum fixture. It reproduced locally; both listener cases now use the existing bounded reservation helper. All 41 credential tests pass on rerun. One intervening local run hit a separate ambient bind collision in the two-occupied-port case. - The full local `pnpm test:run` attempt was stopped after host contention caused focused-suite timeouts. The affected focused tests passed on rerun. An expiring trace fixture and a missing private-conversation state were corrected. The successful full CI run is the complete-suite verification. - Hosted Interrupt previously cleared the original queue and produced exactly one successor with neutral interruption feedback. It exposed the dispatch authorization defect. Retry on that earlier build was rejected for missing `agents:create` before creating another run. - Deployed the final application build (`38257f391`) to the scoped hosted instance and verified readiness. The latest PR commit changes only the credential test fixture; application code matches that deployment. A live Retry by the same operator without `agents:create` created one successor attributed to that operator, passing the former dispatch permission gate. Startup then stopped at `configuration_incomplete` because that operator has not configured their required personal Claude Code OAuth secret; the post-deployment run page confirms the operator identity and no provider work started, and the My secrets UI still shows the token as not set. Provider execution remains unverified pending that credential. No permission grants or credentials were changed. ## Risks Queue admission and finalization can race. The task lock, durable queue receipt, current comment IDs, and successor guard prevent duplicate dispatch. Process and lease stop checks, task pauses, approvals, ownership, and budgets remain in force. The API change only allows null on legacy Interrupt; native steering still requires an active run. No schema migration is required. Active non-viewer board members can now invoke existing agents without agent-creation permission. Agent self-invocation rules, raw provider-trace admin access, task retry scope, external chat authorization, and action-specific user/agent permissions remain enforced. ## Model Used OpenAI GPT-6 through Codex. The session does not expose an exact backend model ID or context-window size. Used reasoning, repository search, code execution, tests, and browser tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-12 17:11:30 -05:00
DottaandPaperclip	0e14c61da7	fix: fence native startup against cancellation (#13316) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Users can pause a task while its runner prepares to start. > - Cancellation must prevent preparation from creating new execution authority. > - Native runtime selection could run after cancellation and leave an unclaimed recovery coordinator. > - Saved user messages then waited for recovery that had no eligible worker. > - This pull request fences startup and lets explicit user continuation settle verified, unclaimed startup state. > - Tasks can continue after cleanup while keeping provider ownership and execution safeguards. ## Linked Issues or Issue Description Refs #13285 for startup controller leases and #13270 for explicit continuation and saved-message recovery. Related #13293 covers retained processes that actually started; this change covers cancellation before the native provider claim. Related #13315 covers legacy queued-message delivery. **What happened?** Pausing a task during startup could cancel its heartbeat before native runtime selection. Stale preparation then created an observed native coordinator on the cancelled run. The coordinator had no provider result or eligible recovery worker. A later Continue message stayed queued indefinitely. **Expected behavior** Cancellation fences native startup. After verified cleanup, a newer user message starts one fresh conversation turn. An unverified execution keeps its hold and a clear explanation. **Steps to reproduce** 1. Start a task with the native runner and delay startup preparation before runtime selection. 2. Pause the task, then release preparation. 3. Resume the task and send Continue. 4. Before this fix, native selection can persist after cancellation and block the saved message. 5. Repeat from persisted cancelled startup state after a server restart. **Paperclip version or commit** Reproduced against master at `586b5ec82` with isolated PostgreSQL regression fixtures. **Deployment mode** Self-hosted server built from source, with Paperclip Runner. ## What Changed - Serialize the cancellation fence and native runtime selection on the run row. Revalidate the startup controller lease. - Refresh the runtime before dispatching cancellation, and reject terminal or cancelled runs at the native provider claim. - Recognize never-claimed coordinators only after startup and environment cleanup are verified. Reject process, provider, owner, and conflicting launch evidence. - Settle that coordinator atomically with a new authenticated user turn. Preserve history, unknown outcomes, and attempt counts. - Reuse the saved-message worker after restart and retain pause, budget, approval, and ownership gates. - Add startup, restart, duplicate-admission, and negative-proof regressions. Document the rule. ## Verification - All 687 tests pass across the complete heartbeat recovery, explicit continuation, and native session executor suites on the rebased branch. - Six focused race regressions also pass: cancellation before and after native selection, Stop racing adapter registration, process termination during a database failure, and a run finishing during cancellation. - `pnpm -r typecheck` and `pnpm build` pass after rebasing on master at `ab15aff39`. - Greptile is 5/5 on `f3dea2ab27facdf0360b56172ebd3e5219e25538`, with no open review threads. Policy and security checks pass. - All 32 CI checks pass on the final head, including the full server/workspace test matrix, all three browser shards, runner verification, typechecks, build, and canary dry run. The two optional Storybook jobs are skipped. [CI run](https://github.com/paperclipai/paperclip/actions/runs/34697945514). - Local full-suite limitation: the broad `pnpm test:run` attempt reported two failures outside the changed area after unusually long test durations (about 65 seconds for supporting skill-file saves and 933 seconds for setup-token login). Both cases passed isolated reruns, with no code changes. The broad local run was stopped after CI completed successfully; no clean full local-suite pass is claimed. ## Risks Cancellation and startup overlap. The run and coordinator locks provide the authority fence; cleanup and process evidence provide the containment proof. Historical runs without sufficient evidence remain blocked. A saved user message authorizes a fresh turn, not automatic replay. No schema migration or dependency change. ## Model Used OpenAI GPT-6 through Codex, using reasoning, repository inspection, code execution, and tests. This session does not expose the exact backend revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-12 11:39:36 -05:00
DottaandPaperclip	9031516a7e	fix: recover legacy Daytona startup failures from task and inbox (#13272) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Legacy conversation adapters can run in Daytona sandboxes. > - A server restart during provisioning can occur before the invocation event exists. > - Recovery then lacks the old adapter identity and leaves a hold that ordinary user retries cannot clear. > - A remote launch can also fail when its host relay looks for Node in the sandbox PATH. > - This pull request records the adapter at claim time and restores explicit user continuation after verified cleanup. > - Users can recover from the task or inbox while the failed run and uncertain action history remain intact. ## Linked Issues or Issue Description Refs #13237, #13239, #13254. Those changes cover recorded conversation runs, native user continuation, and explicit remote Stop. This change covers legacy failure before `adapter.invoke` and exact task/inbox Retry. Refs #9771 for overlapping generated-command quoting. This change also supplies the absolute host Node executable. Refs #13163 and #13264 for the separate native restart and retained-workspace work. **What happened?** A legacy Daytona run interrupted during provisioning became `process_lost` without an invocation event. Recovery preserved an execution hold, and Retry or a new task reply could not resume it. Cleanup could also run before the Daytona plugin was ready. On a macOS host, a subsequent ACP relay launch failed with `env: node: No such file or directory` because the remote launch environment did not contain the host Node path. **Expected behavior** An interrupted conversation can continue after its previous execution stops. Explicit Retry and new user replies should start a fresh turn with the task history. Cleanup failures must remain visible and recoverable. The host relay must use the host Node executable. **Steps to reproduce** 1. Use a legacy Claude adapter with a Daytona environment. 2. Interrupt the server after it acquires the sandbox lease and before it records `adapter.invoke`. 3. Restart and inspect the task hold. 4. Retry from the task or inbox, or send a new task reply. 5. Confirm the old sandbox has stopped and one new response arrives. **Paperclip version or commit** Reproduced from master at `3bafac12f796fbea02e609e1074a9639f872e9c4`. The branch is rebased on `51b0e01ea`, including #13261 and #13270. **Deployment mode** Built from source on macOS with a real Daytona sandbox and the legacy Claude ACP adapter. ## What Changed - Count new browser specs with the scheduler's median duration in the shard-balance check. This fixes a false policy failure after new specs arrive from both branches. The balance threshold is unchanged. - Persist server-owned adapter identity in the queued-to-running claim before provisioning starts. - Wait for provider plugin startup before restart cleanup. Keep failed cleanup leases as active ownership blockers. - Admit exact board retries and new user comments after verified termination. Retain the old run, task history, approvals, and unknown action outcomes. - Adopt repeated Retry requests. Permit one scoped cleanup attempt per explicit user Retry after the automatic limit, with an activity record. A later user Retry can recover after a transient provider failure; automatic attempts remain capped. - Resume replies deferred during cleanup, including historical legacy startup failures. - Launch the host ACP relay through the absolute host Node executable. - Add a task-level Retry button and return actionable blockers when retry admission is refused. - Add database regressions and three browser recovery journeys. Exclude installed third-party dependency skills from the shipped-skill audit. ## Verification - Current head: `d23c84181`, rebased on `51b0e01ea`. Conflict resolution retains the saved-message recovery, local stop receipts, and wait reasons from #13270 alongside exact legacy Retry support. - Real Daytona: interrupted the server after lease acquisition and before adapter invocation. Restart cleanup confirmed provider termination. Task Retry cleared a seeded historical hold and a real Claude agent returned `Recovery verified.` in the task. Removed the disposable sandbox and environment after testing. - All three browser recovery journeys passed again after the final rebase. Task Retry, Inbox Retry, and a new reply each produced one fresh successor, completed the task, preserved the failed run, and retained the answer after reload. - All 29 e2e/server shard-partition tests passed. The balance check now uses the scheduler's median fallback for unmeasured specs, with the same balance threshold. - Server typecheck passed after rebuilding the generated runner dependencies. The combined recovery/route run passed 136 of 137 tests. Its remaining route test timed out during the first cold module import at its explicit 10-second limit; an isolated rerun reproduced that timeout and passed the other 51 route cases. The complete CI suite passed on this head. The same route file passed all 52 cases in CI, including the first cold import in 7.5 seconds. - Before the final rebase, recursive typecheck, full build, UI token gates, 132 targeted server tests, and the complete [CI workflow](https://github.com/paperclipai/paperclip/actions/runs/34650004085) passed. The subsequent CI failure was the shard-balance accounting mismatch fixed here. - Greptile reviewed `d23c84181` at 5/5 with no outstanding actionable findings. The complete [current CI workflow](https://github.com/paperclipai/paperclip/actions/runs/34653327949) passed on attempt 2. All test, typecheck, build, and canary jobs passed on the first attempt. Docker setup timed out fetching BuildKit from Docker Hub; retrying that job and its dependent aggregate succeeded. ## Risks - Recovery admission changes executable authority. Company, task, agent, user, approvals, process ownership, and provider termination checks remain required. - Explicit continuation starts a fresh conversation with history. It does not certify unknown external action outcomes or rerun non-conversation adapters automatically. - Changing task status alone does not clear an execution hold. The task now offers an explicit Retry action. - Historical adapter claims and invocation events take precedence over current agent settings. Known process or webhook runs retain their hold. Pre-upgrade rows with no adapter evidence may receive only a new explicit user turn after termination proof; they do not become eligible for automatic replay. - No schema migration or sandbox-image change is required. This branch has not been deployed to production. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution, browser automation, and test execution. The exact deployment model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-11 18:08:14 -05:00
DottaandPaperclip	51b0e01ead	fix: resume saved user messages after execution recovery (#13270) ... Preserve verified native process-stop evidence and retry saved user messages through normal continuation admission after recovery cleanup. Show the current wait reason and serialize delivery so a saved message starts one fresh turn. Validated with 410 focused tests, typecheck, build, token gates, all PR CI checks, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-11 17:12:55 -05:00
DottaandPaperclip	663c44cb2b	fix: continue conversations after confirmed remote runner stop (#13254) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Users can stop a run and send another message on the same task. > - Remote runners need evidence from their sandbox provider that execution stopped. > - Local process checks cannot prove that a remote process exited. > - This pull request records provider stop receipts and uses them for conversation admission. > - New user messages can proceed after confirmed cleanup without repeating interrupted actions. ## Linked Issues or Issue Description Refs #13237 and #13239. Related: #13163 covers app-restart recovery; this change covers an explicit stop followed by a new user message. **What happened?** A stopped remote Claude ACP task kept its execution hold after Daytona cleanup succeeded. Native runners also rejected remote process identities and retained stale session cleanup gates. A message sent during cleanup could stay deferred after the sandbox stopped. **Expected behavior** After the provider confirms that the old execution stopped, a new user message starts a fresh turn. Pending user messages must not need another message to trigger admission. Prior action outcomes remain recorded. **Steps to reproduce** 1. Start a long-running task in Daytona with a legacy Claude ACP or native ACP runner. 2. Cancel the run while its tool is active. 3. Send a new message immediately, or after cleanup completes. 4. Observe the execution hold despite the old sandbox having stopped. **Paperclip version or commit** Reproduced on master at 7b829efdf6. The branch is rebased on current master. ## What Changed - Add optional provider stop receipts to sandbox release and destroy hooks. Old plugins remain compatible. - Bind receipts to the company, run, lease, and provider resource. Failed cleanup cannot supply stop authority. - Acknowledge legacy remote cancellation after confirmed termination. - Admit native user continuations using remote receipts instead of host process checks. - Retire only the settled cleanup owner matching the stopped company, run, and provider resource. Isolate cleanup gates between remote sandboxes, including two sandboxes owned by one run. - Reconsider user messages deferred during remote cleanup through normal admission, including successful later cleanup retries. - Preserve receipts through cleanup retries and inline cleanup after failed startup. - Permit provider destruction after a terminal remote checkpoint failure. Busy ownership still blocks destruction. - Add regression tests and update execution semantics. - Give the real preview fixture ten seconds for cold startup. Reuse release-mode Rust artifacts for the filtered parity checks, avoiding a duplicate debug test build that exhausted CI disk twice; test filters and assertions are unchanged. ## Verification - Recursive typecheck and build passed. - Targeted server, Daytona plugin, and native runtime tests passed. They cover missing or mismatched receipts, failed cleanup, local process protection, exact cleanup ownership, and a message sent during cleanup. - Live Daytona tests passed for legacy Claude ACP, native per-turn, native warm, and a newly created native runner using disposable sandboxes. Each original run was cancelled; its explicit follow-up completed with no execution hold. The disposable case confirmed a new sandbox after deletion. - Native tests used the provider's Opus 5 selector, `opus[1m]`, and the Linux runner bundle from the sandbox image. Existing checkpoint/sync finalization warnings remained visible before the native follow-ups reached committed success. This change does not repair those separate warnings or guarantee recovery of uncheckpointed files. - A direct live provider test also passed with the final delete-wait change: the destroy hook returned its receipt only after Daytona reported the sandbox destroyed. The final Daytona plugin suite passed all 153 tests; its build passed. - After rebasing on master, 284 targeted server/plugin tests and 313 native executor tests passed. The native session runtime suite passed all 128 tests. - The review regression passed all 145 tests across the continuation, environment runtime, and pending-cleanup sweep suites. Recursive typecheck and build passed again after that fix. - The security review's exact-resource finding is fixed. Cleanup completion requires the same provider-resource scope used during session creation. All 128 native runtime tests, 61 server continuation/cleanup tests, recursive typecheck, and build passed after this fix. The two-sandboxes-in-one-run regression proves one receipt cannot retire the other quarantine. - Greptile reviewed the final commit at 5/5, the security scan passed, and no review threads remain unresolved. The final native executor suite also passed all 313 tests. - The full local suite reached 10,616 passing tests before stopping on four failures. All four now pass in focused reruns: the final-code continuation/teardown suites, the attachment suite, and the native session test after building its required fake-provider binary. The original full run overlapped source edits and did not reach the remaining groups; it is not counted as a full-suite pass. - The preview exposure suite passed all 25 applicable tests (three Linux-only tests skipped locally) after the startup allowance change. Both release-mode Rust parity commands passed locally. All final-head PR checks passed, including full server/workspace/browser test groups, full native runner verification, build, typecheck, canary dry-run, and the aggregate gates. ## Risks - A provider must return a receipt only after confirmed termination. Incorrect provider claims could permit overlapping execution. - Older providers without receipts retain the existing hold. Missing evidence, failed cleanup, active ownership, pauses, approvals, and budgets still block admission. - This change preserves unknown action outcomes and old checkpoints. It does not authorize replay or alter historical runs. - No database migration or telemetry contract change is required. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, code execution, and browser tools. The exact backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-11 15:18:37 -05:00
DottaandPaperclip	5545f6d166	feat: let user messages continue stopped native tasks (#13239) ... ## Thinking Path > - Paperclip manages AI agents and their tasks. > - A failed native run can leave a durable execution hold. > - The hold prevents automatic replay of actions with unknown outcomes. > - It can also prevent the agent from answering a new user message. > - A new user message should authorize a fresh turn after the prior execution stops. > - This pull request adds that admission path and preserves the existing execution gates. > - Users can continue the conversation without certifying every past action. ## Linked Issues or Issue Description **Subsystem affected** Server task wake admission and native execution recovery. **Problem or motivation** A native task can remain blocked after automatic recovery stops. A new user message is saved, but its run is cancelled before the agent can answer. **Proposed solution** Use a new authenticated user comment to authorize a fresh turn. Check stopped predecessor ownership and available history. Retain uncertain action outcomes. Commit the new run and hold retirement together. **Roadmap alignment** This is a focused improvement to the existing self-healing runs and recovery behavior. Builds on merged #13237, which covers legacy conversation continuation. This PR adds native admission and preserves native automatic-recovery eligibility and budgets. ## What Changed - Admit a fresh native turn for a new user comment after every held predecessor has stopped. - Validate the comment author, task, timing, process ownership, controller, and cleanup leases. - Preserve failed runs, unknown action outcomes, and the failed incident's attempt count. - Record the new comment and run in the existing recovery audit history. - Validate the saved continuation source and discard consumed user-wake authority from later automatic replacements. - Keep pause, approval, budget, ownership, and dependency interaction rules. - Add database and actual wake-path regressions. Update the execution contract. ## Verification - [Full CI run 34626750213](https://github.com/paperclipai/paperclip/actions/runs/34626750213) passed on `c58e6c087e0df7530c747d80b27d491da925a9c4`: all 31 reported checks passed, including all server/workspace suites, browser shards, native runner verification, build, typecheck, release dry run, and aggregate gates. The two conditional Storybook checks were skipped. - Greptile reviewed this exact head at 5/5. All review threads are resolved, and security checks passed. - All 218 local targeted tests passed across explicit native continuation, continuation history, safe replacement, durable chat wakeups, wake queue, issue liveness, native session resume, and run dispatch. The native implementation is unchanged by the final rebase onto master. - Full workspace `pnpm -r typecheck` and `pnpm build` passed on the final head. Complete test coverage is supplied by the green CI suites; local tests used the targeted suites above. - Regressions cover scoped authorization, concurrent delivery, live ownership, later admission gates, retained message receipts, and automatic replacement after terminal-task or reviewer changes. ## Risks - A fresh model turn can choose to repeat an action. Paperclip preserves prior history and does not replay recorded calls. - Missing process identity and remote ownership without a target-aware stop proof retain the hold. A terminal database row alone does not prove that execution stopped. - No schema or dependency changes. Existing historical tasks are not awakened by deployment. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and test execution. This session does not expose a more specific model build ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-11 12:33:57 -05:00