PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-12 14:19:00 +01:00

Author	SHA1	Message	Date
Dotta	a3e7e63330	Guard Native recovery with process-space proof and explicit containment (#19813 ) ## Thinking Path > - Paperclip manages agents and their work. > - Recovery must preserve execution ownership and accepted results. > - A saved PID proves nothing outside its recorded process namespace. > - Uncertain unfinished work must stay held until its original owners stop. > - Operators need a source-bound containment action and a separate ordinary Retry. > - Accepted results must use the existing finalizer without another provider turn. > - This PR keeps those paths safe, recoverable, and quiet between real obligations. ## Linked Issues or Issue Description Refs: #19713 Refs: #19591 Recovery could use a local PID to release or replace a controller from another process namespace. Connected-chat Retry also rejected successful containment on a task in `todo`. The new containment action used a broader permission than the existing owner-stop action. A later review found that result-less guards also rejected the existing accepted-result confirmation action. ## What Changed - Record and verify the controller's kernel/PID namespace. Keep unknown and foreign unfinished ownership held. Preserve independent durable stop evidence and the existing remote lease fences. - Use the shared accepted-result ownership reconciler. Bind settlement to the observed lease token, generation, boot, PID, namespace and process birth. Retain finalizer, cleanup, current-controller, live-owner and remote exclusions. - Share the remote-owner reader across recovery consumers. Scope runner leases to the same company, task and run. Do not make local PID probes for remote accepted-result ownership. - Gate original-host containment on `runtime:confirm_owner_stop`. Bind it to the current task, source, assignee, original expiry, lease and Stop state. Audit separate controller/provider containment and reviewed action outcomes. - Keep containment separate from ordinary Retry. Use one canonical committed receipt and the normal admission, budget, workspace and lineage checks. Preserve the recorded external-action outcomes. - Keep the controller-only confirmation actionable for both result-less work and accepted results. Result-less work gets its next source-bound attention in the same transaction. Accepted results keep their original result and use normal shared settlement/finalization. - Keep identity inspection quiet. Preserve the original foreign-owner expiry and use the existing scoped finalizer in accepted-result recovery controls. - Update the recovery card, validators, execution projection, docs and access maps. Reuse existing canonical columns; add no migration or ownership backfill. ## Verification Current publication head: `71ea76b568de09d6313d8992e803131d9d5c54f3`. The current fix has a causal red/green check. On the test-only revision, both foreign-local stopped/fenced controller cases failed because recovery probed saved runner, process-group and provider PIDs in this host's namespace. This measured the wrong probes; it did not run duplicate providers. After the fix, the same two cases passed every no-probe, no-claim, no-controller-stamp, no-stop-receipt and unchanged-authority assertion. Three existing remote preparation, grace and fenced-controller progress cases also passed. Total: 5 passed, 0 failed;170 filtered rows excluded. All three normal affected App TypeScript programs passed on this head (660,681 and428 roots). The fix separates a controller wait from execution stop proof. Independent live/grace evidence still schedules recovery. A stopped/fenced foreign controller can pass this namespace guard only for remote execution, which keeps its later checkpoint and never-launched checks. Foreign local execution stays held before runner or provider PID inspection. Earlier source-scoped results remain evidence, not fresh executions on the current head: - On `c86dd741`, the complete failed-owner file passed 25/25, with no skips or filters. Empty scans wrote no controller stamps. Foreign ownership kept its original expiry. Normal returned-claim settlement and attention at the exact deadline passed. - On `c86dd741`, 15 selected accepted-result/restart controls passed. The81 filtered rows are excluded. Both completion cases used the existing scoped finalizer and kept their accepted result without another provider turn. - Trusted CI at synthetic merge `50d072a5` (actual master `1b7fe334` plus `c86dd741`) passed the complete 17-case maintenance-server file. This includes the original90-real-second quiet observation and two qualified hourly windows. The quiet case took118,918ms. The four jobs that failed on the earlier publication all passed on that synthetic merge. - On `6243b32c`, the complete ownership-report file passed 29/29. Native NULL namespace stayed unprovable and generic compatibility remained. Three selected recovery controls passed;170 filtered rows are excluded. - On `f81f2c3c`,356 business cases passed: whole ownership-route221, recoveryUI70, owner-proof38 and controller-confirmation8; selected accepted-result authority18; and one real connected-chat containment, separate authorized Retry, useful Native result and downstream reply case. The2,024 filtered rows are excluded. A separate earlier accepted-result causal case kept the original result and completed after restart without replay. Normal Shared exports, App programs, UI typecheck, token gates and map checks passed on their recorded revisions. Unchanged DB, adapter, Eval and Runner inputs/outputs have explicit carry proofs. The clean normal merge with fetched master `8fe524f7` preserves all31 exclusive owned paths and14,827 unrelated master entries. The nine overlapping paths were reviewed, including the upstream unidentified-run stop action. Fresh CI must validate that actual merge composition. PG cases used existing external PG17, fresh owned databases, pools at most4, one worker and no SQL observer during cases. Registered clients and recorded process groups closed. Independent final SQL audits found zero campaign sessions before normal database cleanup. Original failing fixtures remain retained. Original failures are preserved. One new test first failed compilation due to a missing getter import; an import-only correction passed its full owning programs before the causal red/green check. A local quiet attempt stopped in the normal prerequisite hook before any business case and its collector rejected two normal build markers. The later17-case CI pass establishes quiet recovery; it does not turn that failed local collector into a pass. An earlier useful case passed before its private cleanup collector failed on an owned read-only directory; normal cleanup completed after all owners closed. Earlier runner-shutdown cancellation and a separate private whole-suite failure remain unexplained. Current-head checks are complete:57 success,1 neutral and3 skipped, including the successful aggregate CI gate. The legacy status passed. Greptile is5/5 and Ponytail gives an explicit5/5 Ship verdict for this head. All seven review threads are resolved. Full recursive typecheck, the full test suite and release build were not repeated locally. No result here proves deployed suspension, a live provider or termination on another host. ## Risks - Unknown physical ownership can require an owner/admin action. A false containment statement can be unsafe. - New work, reassignment, Stop, renewed authority, accepted results or pending cleanup can invalidate an action. The operator must refresh. - Controller confirmation does not prove provider/tool containment. Completed external actions must not be replayed. - Generic legacy and child-process compatibility remains separate from the strict Native controller policy. - Existing saved state and canonical schema are preserved. Permanent safety refusals remain held with their normal named recovery path. ## Model Used OpenAI `gpt-6.1-sol` agents implemented and tested the changes with reasoning, repository tools and code execution. A GPT-6-based Codex managing agent reviewed and integrated them. The host does not expose the manager's exact build or the context-window sizes. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-09 12:06:11 -06:00
DottaandClaude Opus 5.5	365c771030	fix(chat): give project and workspace tool results their page links (#20324 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People ask a chat agent to take them to a project or to the files of a task > - The agent finds the right project or workspace, but it cannot link the page > - The `list_projects` and `list_workspaces` tool results have no page route, and the chat link guidance names no project or workspace format > - The agent then writes a bare path or a shortened directory that the person cannot open > - This pull request adds the page `href` to these tool results and adds the project link format to the chat guidance > - The benefit is that chat replies give a clickable link to the real project or workspace page ## Linked Issues or Issue Description What happened In chat, a person asked "Take me to the Launch project and show its open tasks." The agent found the project. Its reasoning said "There's no project page route documented explicitly". The reply wrote the project as a bare path `/COM/projects/<id>`, which is not clickable. In a second chat, a person asked where the files of a task are. The agent named the correct workspace but gave a shortened directory and no link. Expected behavior The reply links the project page (`/<prefix>/projects/<id>`) and the task's workspace page (`/<prefix>/execution-workspaces/<id>`) as Markdown links, so the person can open them. Steps to reproduce 1. Create a company with a project and a task that has a saved execution workspace. Enable isolated workspaces. 2. In chat, ask the agent to take you to the project, or to say where the task's files are. 3. Look at the reply. It has no project or workspace link. Related: Refs #17985 (an earlier open PR that adds project and goal link wording for newly created projects only). This PR covers project discovery and workspaces and adds tool-result hrefs. ## What Changed - `list_projects` rows now carry `href: /<prefix>/projects/<id>`. Both the native runner tool authority and the MCP project-tools route pass the company prefix. - `get_workspace` and `list_workspaces` now carry `href: /<prefix>/execution-workspaces/<id>`, only while isolated workspaces are enabled. The workspace route redirects to the dashboard when the feature is off, so no href is offered then. - The chat link guidance (`AGENT_CHAT_DIRECTIVE`) adds the project link format. It tells the agent to link a returned `href`, and to name a workspace that has no `href` instead of linking it. - Regression tests: guidance text against the registered UI route, `callProjectTool` href, and an embedded-Postgres authority test for project hrefs and the workspace flag on/off. ## Verification - Red then green: the three new tests fail on `origin/master` source and pass with this change. - `npx vitest run` in `server/` for: `agent-conversations`, `task-workspace-api`, `project-tools-pagination`, `task-workspace-tools`, `chat-project-tools`, `dot-runner`, `heartbeat-list` (prompt-size limits), `task-markdown`, `muse-native-visibility`, `paperclip-runner-tool-authority`, `runner-api`, `codex-local-execute`, `board-access-denial`. All pass. - `npx tsc --noEmit -p server` passes. - Live chat eval: see the comment on this PR for the paid runner E2E run of `navigation-preferences.find-project-view` and `workspaces.locate-files` with this change. ## Risks - Low risk. The change adds one optional field to read-only tool results. The output schemas allow extra fields. - The guidance grows by two sentences. The prompt-size test in `heartbeat-list.test.ts` passes. - `list_projects` and the workspace tools now do one extra small read (company prefix, and the instance experimental settings for workspaces). ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 11:52:44 -06:00
Devin FoleyandPaperclip	0f84df91da	fix(native): accept a completed child's file in the parent's completion report (#20325 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - In onboarding, the agent plans the user's first task, creates one child task for the work, and waits for it > - The `first-task` skill tells the agent to publish a requested file on the child task. When the parent wakes, the agent links the child's file and does not publish a copy > - Native `register_deliverable` is bound to its current task, so the parent cannot register the child's file again > - The native completion check required each cited attachment to be on the task being completed > - So when the parent cited the child's download link, `paperclip_finish` failed, and the agent reported the finished onboarding task as blocked > - This pull request lets a completion report cite an attachment on a direct child task that is done > - The benefit is that onboarding closes after its child task delivers a file, and the user does not see a false "blocked" status ## Linked Issues or Issue Description Related: #18258 (onboarding artifact completion instructions). What happened? An onboarding run created a child task to build and publish a project. The child task registered `project.zip` with `register_deliverable` and was marked done. The parent woke with reason `issue_children_completed`. The agent called `paperclip_finish` with `reportedWorkDisposition: "done"` and cited the child's `/api/attachments/<id>/content` link. The call failed two times with: ``` Completion cites no registered attachment on this task. Use register_deliverable for the requested file and cite deliverable:<attachmentId> from its receipt. ``` The skill does not let the agent register the file on the parent, so the agent called `paperclip_block` with "Paperclip completion validation" as the owner. The parent task stayed `blocked`. Nothing wakes it again. Expected behavior The parent completes and links the child's file. Steps to reproduce 1. Create a task with a child task. 2. In a native run on the child, register a file with `register_deliverable`, and mark the child done. 3. In a native run on the parent, call `paperclip_finish` with `reportedWorkDisposition: "done"` and the child's `deliverable:<attachmentId>` or `/api/attachments/<attachmentId>/content` as evidence. 4. The call fails with "Completion cites no registered attachment on this task". ## What Changed - `server/src/services/native-runtime/native-deliverable-feedback.ts`: when a cited attachment is not on the task, `validateNativeDeliverableEvidence` also looks for it on a direct child task with status `done`. If it is there, the citation is accepted. - The child's file is context, the same as an attachment from an earlier run on the same task. It does not count as this run's publication of a file that the parent's own request names. - With `lockSources`, the finalization path locks the attachment and child task rows with `FOR SHARE NOWAIT`, as it does for other sources. - The error text now says "on this task or its completed child tasks". - This applies to the `paperclip_finish` check and to the two finalization checks, because they use the same function. - `doc/AGENT-ARTIFACTS.md`: the native runner section states that a completion report can cite a file on a completed direct child task. ## Verification - Extended "prepares one verified same-run attachment and replays without duplicates" in `server/src/services/native-runtime/native-runner-file-handoff.test.ts`: - An attachment on an unrelated task is still rejected. - An attachment on a child task that is not done is still rejected. - An attachment on a done child task is accepted, through `paperclip_finish` and through the locked finalization path. - When the parent's own request names a file, the child's file alone does not satisfy it. - Without the source change, the test fails with the same error as the run above. - `npx vitest run` on `chat-attachment-reuse`, `native-completion-feedback`, `native-completion-format-feedback`, `native-deliverable-feedback`, and `native-runner-file-handoff` tests in `server/src/services/native-runtime`: 281 passed. - `npx vitest run src/__tests__/native-result-commit-work.test.ts src/__tests__/native-status-arbiter-corpus.test.ts` in `server`: 167 passed. - `npx tsc --noEmit -p server/tsconfig.json` passes. ## Risks - Only direct children with status `done` are accepted. A grandchild's file, or a file on a child that is not done, is still rejected. - A parent that names a requested file in its own request still needs its own delivery evidence. This keeps the current rule for earlier outputs. - A task that is already blocked by this error does not change. The user can mark it done, or comment so the agent runs again. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (code reading, database queries, shell commands, and test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-09 09:44:17 -08:00
DottaandClaude Opus 5.5	47dccbf3eb	fix(recovery): re-prove an exhausted save's agent stop and free the workspace (#20295 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When an agent run ends, Paperclip saves the files that the run changed in its remote workspace. Until that save settles, the run holds the workspace, so no other task can write it > - When the save fails, a bounded system recovery proves that the agent stopped (a provider stop receipt) and then saves the files again > - If the stop cannot be proven before the 30-minute recovery window ends, the recovery is exhausted with `agent_stop_unconfirmed`. Exhaustion was terminal: Paperclip never proved the stop again, even after the run ended and its sandbox had a stop receipt > - The hold then stayed forever. On staging, a child task's save was exhausted this way on Nov 5. Its parent and its two sibling tasks share that workspace, and all of them stayed blocked for four days > - This pull request re-arms such an exhausted save automatically, on a bounded backoff, once the run is provably over. After the bounded re-arms, the hold continues from the last save with the unsaved files attached > - The benefit is that a dirty workspace hold always settles without a person, and no unsaved work is discarded silently ## Linked Issues or Issue Description Refs #20273 (the stall sweep stops failing wakes against this kind of hold) Refs #20278 (a settled restore hold returns its warm sandbox) Refs #19860 (subtasks get their own workspace) What happened? A child task's run wrote its remote workspace and failed to save it (`native_workspace_sync_out_retry_exhausted`). The save recovery made one attempt. That attempt failed with `agent_stop_unconfirmed`, and the recovery window ended before a second attempt. The recovery became `exhausted`. Ten minutes after the first attempt, the sandbox lease got a provider stop receipt, so the agent was provably gone. But nothing looked at the exhausted recovery again. The run kept the workspace's writer hold. The parent task and every other task on that workspace could not start ("The workspace still has an unfinished writer or save operation."). The blocker told a person to press Retry. Expected behavior An unproven stop is a wait, not a refusal of the saved files. When the run is over, recovery must prove the stop again and finish the save, so the hold settles automatically. If the stop still cannot be proven, Paperclip must keep the unsaved files and then release the hold. A person must not need to act. Steps to reproduce 1. Run a task in a remote (sandbox) workspace. Make its final save fail. 2. Keep the workspace recovery from proving the agent stop until its 30-minute window ends (for example, keep the finalization ownership busy). 3. See that the recovery action is `exhausted` with `agent_stop_unconfirmed`, and that it stays so after the lease gets a stop receipt. 4. Wake another task bound to the same workspace. See that its setup fails because the workspace is held. Root cause - `server/src/services/workspace-recovery.ts` (`recoverRetainedWorkspace`): an `exhausted` state only goes to the sibling settlement and to the retained-source fallback. The fallback accepts only permanent source refusals (`isUnverifiableSourceFailure`). Every other exhausted state is handed to the board and never attempted again. - `recheckStoppedWorkspaceRecovery` skips `exhausted` states, and it only accepts acknowledged cancellations. A failed run whose lease got a stop receipt later is never re-examined. - `retainedWorkspaceCandidates(…, obligationsOnly)` and `nextWorkspaceRecoveryAt` do not select exhausted `agent_stop_unconfirmed` holds, so the recovery worker never wakes for them. ## What Changed - New `server/src/services/workspace-recovery-stop-proof.ts` (`rearmExhaustedStopProof`). For an exhausted save whose last failure is `agent_stop_unconfirmed` (`recheck_evidence`), it starts a new bounded system recovery generation. It does this only when the source run is provably over: the run is terminal, no local execution remains, and no controller lease or finalizer lease is live. The new generation repeats the strict stop proof (a provider stop receipt) and the save. It records an `issue.workspace_recovery_rearmed` system activity. - Re-arms are bounded: at most 4. The first is due at once. The next ones are due 30 minutes, 2 hours and 8 hours after the previous re-arm. The `stopProofRecovery` marker in the recovery action evidence records the count and the next due time. - `workspace-recovery-fallback-discovery.ts`: new discovery conditions (`stopProofRearmPendingCondition`, `stopProofRearmDueAt`, `stopProofRearmDueCondition`) and `retainedHoldFallbackEligible`. `retainedWorkspaceCandidates` (worker pass) and `nextWorkspaceRecoveryAt` use them, so the recovery worker wakes when a re-arm is due. - `workspace-recovery-fallback.ts`: when every re-arm is spent and the stop is still unproven, the hold takes the existing retained-source fallback. The unsaved files are attached to the task (unapplied, secrets omitted), the unverified epoch is abandoned, and the task continues from the last save. - `system-recovery-state.ts`: the exhausted `agent_stop_unconfirmed` text now says that Paperclip tries again automatically. It no longer asks a person to press Retry. Retry stays available. - Review follow-ups: the re-arm applies to every workspace recovery the worker owns (restore and historical holds too). A due re-arm is scheduled no earlier than the source run's controller or finalizer lease expiry, so the worker is not woken every 15 seconds while a lease is live. A fallback whose bounded checks are spent is reconsidered once when an allocation of the source run changes, and `settleStopOnlyCleanup` wakes the workspace recovery worker when it records a stop receipt. The automatic-retry text is shown only while a re-arm remains. - `doc/execution-semantics.md`: documents the re-arm, its bounds, the fallback, and the upgrade behavior. ### Previous-release story (`rules/upgrade-safe-run-state.mdc`) - Carried forward, no migration. A hold exhausted by an older release has no `stopProofRecovery` marker. The recovery worker treats it as due at once and re-arms it on its first pass after deploy. The new regression test saves the exact Nov 5 evidence shape (`attempts: 1`, `retriesUsed: 0`, `state: exhausted`, no marker) and resumes it with the new code. - Mid-run deploy. A run that is still running, or whose controller lease or finalizer lease is live, is not re-armed. The test covers a live controller lease. The re-arm uses the same CAS (generation and revision) and the same row locks as the other recovery writers, so an old worker's late write cannot overwrite the new generation. - No new refusal codes. No saved run identity changes. The re-arm writes only recovery-action evidence. ## Verification - Red (before the fix), `npx vitest run src/services/workspace-recovery-stop-proof.test.ts`: 3 failed, 1 passed. The staging-shape test failed because the worker did not wake (`nextWorkspaceRecoveryAt` returned `null`) and the hold stayed. The backoff and fallback tests failed for the same discovery gap. - Green (after the fix): 4 of 4 pass. - Staging shape: a parent and 3 children that share one remote workspace. One child's save is exhausted with `agent_stop_unconfirmed`, and its run ended days ago with a stop receipt. Before recovery, the hold refuses the parent (`remoteWorkspaceHoldRefusesStart` is `true`). The recovery worker discovers the hold, re-arms it, proves the stop, saves the files and completes. The workspace is not dirty, it has a new snapshot, and the hold is `free` for the parent. The action stays system-owned and its text asks no person to act. - A live controller lease keeps the hold. No re-arm happens and no lease is resumed. - A re-arm that is not yet due is not discovered. The worker's next wake is in the future, and the text says that Paperclip tries again automatically. - After 4 re-arms, the hold continues from the last save. The unsaved files are captured as a task attachment, and the hold is `free`. - A live controller lease defers the worker's next wake to the lease expiry, and the hold is not selected before it. - A spent fallback is not selected, and a later lease change (a late stop receipt) makes it selected again; the worker then settles it. - `system-recovery-state.test.ts`: the automatic-retry text appears only while a re-arm remains. - Related suites, all pass: `src/services/workspace-recovery`, `src/services/remote-workspace`, `src/services/system-recovery-state.test.ts`, `src/services/task-retry.test.ts`, `src/__tests__/heartbeat-observed-workspace-preservation.test.ts` (34 files, 1510 tests), plus the stop-cleanup suites (`environment-retained-orphan-sandbox`, `live-recovery-work`, `retained-native-session-cleanup-work`, `legacy-workspace-restore-recovery`, native workspace warm-reuse, cancelled preservation, reconstruction and failed recovery; 11 files, 622 tests). - `pnpm verify`: exit 2 ("broader verification required", because `doc/execution-semantics.md` is a global CI input). The focused suites above cover the change, and required CI supplies the broad coverage. - `npx tsc --noEmit -p server/tsconfig.json`: passes. ## Risks - A re-armed generation resumes the stopped sandbox to save its files. This is the same physical path as a manual Retry. It runs only when the source run is provably over. - Re-arms are bounded (4) and spaced, so a stop that can never be proven does not loop. After that, the existing retained-source fallback settles the hold. If the retained files cannot be read, the fallback records why. This is the same behavior as for a refused source. - The parent's workspace receives the child's saved files, because the child (bound by an older release) wrote that workspace. This is the existing #19860 behavior: the files that a legacy child left in the shared workspace stay there. #20314 moves the stranded legacy children to their own workspace. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser read-only API checks) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 11:44:15 -06:00
DottaandClaude Opus 5.5	8fe524f796	fix(issues): let the board confirm an unidentified run stopped instead of an endless Retry wait (#20310 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before a new attempt can start on a task, Paperclip must prove the previous run's agent stopped, so two agents never work the same task at once > - A local native run can be interrupted after it asked to start the runner but before it recorded the runner's process. The runner may have saved state and started a provider, and provider processes can outlive it, so the existing check (#19694) rightly refuses to assume the run stopped > - Today that case has no way out: every Retry is accepted and then waits forever as `process_identity_missing` (storage chaos ST1, t02/t03) > - This pull request makes the wait explicit and actionable. The task explains why it waits, and a board operator who has checked can confirm the run stopped; the parked Retry then starts at the click > - A second shape from the Daytona storage baseline (t06) parked a Retry with no reason at all: a follow-up was interrupted by a graceful shutdown while it was still restoring its workspace. No provider ran, but the restore hold kept the same agent held, so the Retry never started > - The benefit is that no Retry stays accepted-and-waiting with nothing anyone can do ## Linked Issues or Issue Description Refs #19831. Storage chaos wave `w-20261109T145628-s9902` (ST1), tasks t02 and t03. What happened (CHA-3, run `3c3f6241`): - The run requested a runner launch (`native.process_start_requested`), and a K2 restart hit before its process identity was recorded. - After the restart, the resumed attempt failed on the storage outage. The task was blocked. - The board's Retry (202) was parked as `executionWait.reason = process_identity_missing`: "The previous run has no verified stop record." - `recordUnidentifiedLocalRunnerStop` cannot prove the stop because this run's runner saved control-plane state. Once that exists, a provider may have started (#19694), and a provider is not guaranteed to carry `--run-id` or `PAPERCLIP_RUN_ID` (the Codex provider env omits it on purpose). The shared owner prover needs a PID, which the run never recorded. - Nothing could ever clear the wait. Daytona storage baseline `w-20261109T154810-s9911`, task t06: - The run succeeded. The follow-up was cancelled (`continuation_task_ownership_changed`), and the next run was interrupted by a graceful shutdown (`server_shutdown_interrupted`) 3 s into `preparing`. It was restoring the saved remote workspace and had launched no provider. - Restore recovery saved the files (`workspaceRecovery.status: recovered`, system recovery `complete`). But the `legacy_execution_requires_reconciliation` restore hold releases only for a different owner, or for a native Stop fenced before any provider turn, so it kept the same agent held. - The UI Retry returned 202 and parked with `executionWait.reason: execution_recovery`, the generic wait, so the task showed a Retry that never started. Automatic proof needs the runner to record its identity before saving state and to group provider processes under it. That is follow-up runner work; this PR removes the endless wait now. ## What Changed - `server/src/services/native-local-process-stop.ts`: new `confirmUnidentifiedLocalRunnerStopped`. It accepts only a finished native run of the task with no recorded process and whose latest launch evidence is an unconfirmed launch on this host in a local environment. It still refuses while a runner with `--run-id <run>` is running. It records `native.local_process_stopped` with `basis: "operator_attested"` and the attesting user. That is the receipt the existing continuation gate already accepts. - `server/src/routes/issues.ts`: `POST /issues/:id/recovery-actions/confirm-run-stopped` (`runId`, `stopEvidence` of 20 to 12,000 characters). It is board-only and requires issue read plus `runtime:confirm_owner_stop`, like the other stop attestations. It logs `issue.run_stop_confirmed` with the evidence, promotes deferred intents for the issue, and resumes each parked task Retry, so work starts at the click. Refusals return 409 with a code. - `packages/shared`: `confirmRunStoppedSchema`, and the presentation user action `confirm_run_stopped`. - `server/src/services/task-interruption-presentation.ts`: a Retry parked as `process_identity_missing` for a native run that recorded no process now presents as waiting for the user (`canRetry: false`, `userAction: "confirm_run_stopped"`), with a message explaining why and who can act. A run with a recorded process keeps the existing verification wait. - UI: - `ExecutionBlockerNotice` shows the explanation and, for users who can confirm a stop, "Confirm it stopped" with a short evidence form. - It reuses `PreviousServerStopConfirmation`, now with optional `action`, explicit `runId` and `formId`, and a custom button label. - The Inbox names the action. - `server/src/services/execution-blocker.ts`: `interruptedPreparationBeforeProviderWorkCondition`. The restore hold also releases for the same agent when the stopped run was a system-interrupted preparation (legacy runtime never resolved, still `preparing`) with no process identity, no runner or provider events, no finalization or result, and every lease released with a confirmed stop. It mirrors the existing interrupted-preparation proof and still requires the completed workspace-recovery receipt. Auto-replay is unchanged; this only lets the user's Retry or message start. - Review fixes: - `unidentifiedLocalLaunchRefusal` holds the endpoint's launch, host and lease checks, and the presentation offers `confirm_run_stopped` only when it passes. - The attested stop wakes cost accounting in the same transaction. - The confirmation form is keyed by task and run. - The route is registered in OpenAPI. ## Verification Red/green (each new test fails against master and passes with this PR): - `server/src/__tests__/issue-confirm-run-stopped-routes.test.ts`: - The parked Retry presents `confirm_run_stopped` with the explanation. - Confirming records the operator-attested stop and the activity entry, satisfies the stop check, and resumes the parked Retry: its receipt is coalesced into a newly started run. - It refuses with 409 while a runner for the run is running. - Agents and viewers get 403. - `ui/src/components/ExecutionBlockerNotice.confirm-run-stopped.test.tsx`: the notice offers "Confirm it stopped" (and no dead Retry) and posts the run and evidence. A viewer sees the explanation without the action. - `server/src/__tests__/interrupted-preparation-restore-retry.test.ts` (t06 shape): - The restore hold no longer blocks. - A Retry is accepted and resumed into a started run, and the audit hold keeps `replay: blocked`. - The hold stays when a provider turn started, when a runner event was recorded, when an operator stopped the run, or when the sandbox has no stop receipt. - `issue-confirm-run-stopped-routes.test.ts`: - A case per refusal code checks the presentation keeps the generic wait and the endpoint returns 409. - An `alreadyStopped` case. - An accounting-wake assertion. - The UI test switches targets with stale evidence. Regression and static checks: - `ExecutionBlockerNotice`, `WorkspaceExportRecovery` and its control tests: 74 passed. - `tsc --noEmit` passes for `packages/shared`, `server` and `ui/src`. `pnpm check:token-gates` is clean. ## Risks - A board operator can now attest that an unidentified run stopped. The attestation is permission-gated, requires written evidence, is activity-logged, and is refused while a runner for the run is visibly running. A wrong attestation could start a second agent beside an orphaned provider, the same trust model as the existing previous-server attestations. - The confirmation component now accepts a run without a recovery action. Existing callers pass the action as before. - No schema migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 11:36:41 -06:00
DottaandPaperclip	ff2d463eed	feat(cli): manage agent grants and keep remote login on the public origin (#20323 ) ## Thinking Path > - Paperclip lets people control their AI agents and permissions. > - Remote operators need a browser-approved CLI login. > - Proxies can send that login's approval link to an internal host. > - Human member commands cannot grant an agent an explicit permission such as commentary:view. > - This change keeps approval on the canonical public origin and adds Board-only single-grant commands. > - The API checks company access and permission-management authority and records each change. > - An operator can sign in, grant one agent permission, and preserve the agent's other grants. ## Linked Issues or Issue Description What happened? A proxied CLI login returned an approval URL on the upstream host. The signed-in browser was on the public host. The CLI also had no command to add an arbitrary explicit agent grant: member commands manage human members, while agent permissions:update exposes fixed configuration fields. Expected behavior Approval uses the configured canonical public origin. A Board operator with users:manage_permissions can grant or revoke one explicit agent permission without replacing other grants or activating an inactive membership. Steps to reproduce Configure a public auth origin. Create a CLI challenge through a proxy that replaces forwarded-host headers. Inspect the approval URL. Then try to grant commentary:view to an agent through the CLI. ## What Changed - Use the existing public-origin resolver for CLI approval URLs. - Add agent permission:grant and permission:revoke commands. - Add PATCH /api/agents/{id}/permission-grants with strict shared validation and Board-only OpenAPI metadata. - Serialize each mutation on the agent row. Preserve unrelated grants and membership state. Commit the grant and its activity record together and publish activity after commit. - Document remote login and single-grant management. ## Verification - The new approval-link regression failed before the fix and passes after it. - All 25 CLI auth route tests pass. - All 72 affected CLI, login and OpenAPI tests pass. - All 5 PostgreSQL grant tests pass, including actual commentary reads (403 before grant, 200 after grant, 403 after revoke), preservation, audit attribution, concurrent additions, invalid inputs, inactive or missing membership, agent self-grants, cross-company denial, and rollback on audit failure. - Shared and CLI typechecks and server declaration-layer checks pass. - pnpm verify requests broad CI coverage because doc/CLI.md is a global verification input. Greptile reviewed final head `338c58fb24` at 5/5 with no findings. All 58 non-skipped checks pass on that head. CI run: https://github.com/paperclipai/paperclip/actions/runs/42663640985. Live target acceptance is in progress. ## Risks This adds an explicit permission-management API. It requires Board authentication, company access, and users:manage_permissions. Foreign-company agents return 404. Granting a key intentionally makes that key unscoped in its company; revocation removes that key. Other keys and agent configuration remain unchanged. Inactive memberships and unapproved or terminated agents cannot gain grants. No schema change is needed. Revert this change to remove the new management surface; existing grants retain their normal authorization rules. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, shell tools, and browser testing. The runtime does not expose a more specific model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-09 11:35:52 -06:00
Dotta	1b7fe33432	fix(workspaces): preserve import verification across process restarts (#20311 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Online import preserves old local files before the web server can use a disposable filesystem. > - A parent import must authenticate saved child archives before it can omit their directories. > - The operation journal preserves import identity, but completed child checks existed only in process memory. > - A container restart therefore repeated downloads, decryption and restoration that had already succeeded. > - This pull request saves a versioned verification receipt on each immutable child snapshot. > - A replacement process can reuse those checks while it reacquires locks and verifies current local files. ## Linked Issues or Issue Description Refs #19854 for verification reuse within a live source fence. Refs #19873 for cohort import. Refs #20298 for the separate remote-hold admission fix. Searched related workspace verification and restart PRs; none provides durable child verification. What happened? Replacing a server container interrupts a long online import. The original operation and snapshot survive, but a later invocation downloads, decrypts and restores completed child archives again. The current cache ends with the source-lock owner. Expected behavior Completed immutable checks survive process and scratch loss. Resume keeps the same operation and snapshot, reuses only correctly bound verification, and publishes once. Current local files and Git state must still be checked under fresh source locks. Steps to reproduce 1. Import two nested child workspaces, then start the parent import. 2. Kill the importer after a child verification finishes. Remove its scratch directory. 3. Resume in a new process. Before this fix, the completed child is downloaded and restored again. Paperclip version or commit The regression fails on the previous migration verifier. The final branch includes master at `189fefb99183953be37f0d1084275fbb0f35d6e2`. Deployment mode Managed Linux server with PostgreSQL and explicit S3 storage configuration. ## What Changed - Add optional opaque storage namespace identity. S3 binds it to the explicit endpoint, region, bucket, normalized prefix and addressing style. Implicit endpoints retain full verification. - Save a versioned child-snapshot receipt only after canonical crypto, manifest and file restoration succeed and a second HEAD confirms the same strong ETag and size. - Bind reuse to the complete snapshot reference, key material, child lineage, source operation and storage namespace. Require a fresh strong HEAD and canonical validation that the current secret provider can unwrap the saved snapshot key. - Keep current directory/Git capture and source-owner checks on every invocation. Read-only verifier callers do not write checkpoints. - Merge checkpoint metadata under a short row lock without changing parent journal hashes, snapshot retention or update timestamps. Bound reads and writes. Lock timeouts, statement timeouts, deadlocks and serialization conflicts are safe cache misses. Other errors remain fatal. - Extend real PostgreSQL/Git/archive tests with process death, scratch removal, resume, changed bindings and single publication. ## Verification - Baseline RED: two import invocations read each child twice. With durable receipts, each child is read once across both invocations. - Real process regression passed on the final rebased head: SIGKILL the owned importer after one checkpoint, delete all scratch, and resume in a new Node process. It retains the same operation and snapshot, reads each child once across both processes, and records one publication. - 53 focused cases passed in 104.86 seconds on the final production bytes before rebase. They cover existing nested/cohort behavior, namespace identity, changed source/reference/key/lineage/ETag, unknown proof versions, weak validators and bounded row contention. - After clean rebase and stronger scratch/snapshot assertions, the real process case passed again in 18.54 seconds. Server TypeScript, module boundaries and diff checks passed on final head `9da68ee7ce4a778201ef85e21a11c49eeea7bc16`. Required CI supplies final-head broad coverage. - The initial restricted-network dependency setup failed; unchanged preflight passed with normal network access. Initial TypeScript failures in test pipe assertions were corrected. No failed attempt is counted as a pass. - Review found a missing current-master-key check. A real restart regression with all child checkpoints complete reproduced unsafe publication after switching to a different valid master key. Follow-up `7ea92c2abd9ffdb2e7dd81cecf62c8fc7a6e74a5` delegates to the canonical snapshot-key owner before both durable and in-memory cache hits. - The fixed process regressions passed 2/2. Ten key-authority and existing identity/source guard cases passed. The wrong-key process now refuses with zero additional child payload GETs, no publication, and unchanged original journal/snapshot rows. Server typecheck, module boundaries and diff checks passed on the follow-up. - Review also found that transient database failures could make the optional checkpoint cache stop a valid import. Read and save now fall back only for SQLSTATE `55P03`, `57014`, `40P01`, and `40001`. Connection loss, identity changes, crypto failures, programming errors and cancellation remain fatal. Reads use a read-only transaction with one-second lock and eight-second statement limits. - The checkpoint database-fault regression failed two cases before the fix. All four recoverable import cases then passed. The final selected run passed 21 cases, including fatal-error and cancellation controls, real row contention, both process-restart cases and key/identity guards. Server typecheck and boundaries passed. Fault injection is limited to the checkpoint database boundary; the importer, PostgreSQL, Git, archives, storage and publication remain real. - Latest master was merged without rewriting published history. Six real import/process-restart cases passed again on final head `8c8e4213a680ebb5c8d6864416bc3a4f271e1e76` in 38.03 seconds. [Required CI](https://github.com/paperclipai/paperclip/actions/runs/42658248222) finished with 54 successful checks and four intentional skips; Snyk passed. [Fresh review](https://github.com/paperclipai/paperclip/pull/20311#issuecomment-6745354381) is 5/5 with all four threads addressed and resolved. Hosted verification after normal release delivery remains pending. ## Risks - A stale receipt could skip authentication of a different object. Complete reference/lineage/namespace binding, fresh strong HEAD and row-locked save validation prevent that reuse. Missing or unsupported hints trigger full verification. - Mutable local files can change between attempts. Their source locks and fresh captures remain required; a checkpoint never certifies current local bytes. - Checkpoint persistence can contend with other snapshot owners. It has one-second lock and eight-second statement limits; the four classified transient SQL errors retain the normal fully verified path. Other database errors stop the import so lost source ownership cannot be hidden. - No schema, archive format, public API or saved-run identity change. Older snapshots without receipts remain readable and get full verification. Existing data and recovery obligations stay intact. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository tools and code execution. Context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-09 11:03:48 -06:00
DottaandClaude Opus 5.5	0cf70c9d0c	fix(chat): tell overviews to state when the picture is from (#20301 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People ask the CEO chat agent for a quick picture of what the company is doing now > - Such a picture is only useful when the person knows how current it is > - The chat guidance asks for accurate status, but it does not ask for a time > - A live overview said "right now" and gave no time, so the person could not tell when it was observed > - This pull request adds one guidance sentence: state the picture's time as a UTC clock time, from the latest message timestamp > - The benefit is that overviews tell the person when their information is from ## Linked Issues or Issue Description Refs #19614 What happened A person asked the chat agent "Give me a quick picture of what the company is doing right now." The agent read agents, projects, tasks and approvals and gave a correct summary. The summary did not say when it was observed. The only time word was "right now". Expected behavior An overview of current work states when the picture is from, as a clock time. The native prompt already lists each new comment with its timestamp, so the agent has a time it can use. Steps to reproduce 1. Open a chat with the CEO agent in a company that has active, blocked and done tasks. 2. Ask "Give me a quick picture of what the company is doing right now." 3. The reply summarizes the work but gives no observation time. ## What Changed - `AGENT_CHAT_DIRECTIVE` status-update guidance: when the agent describes what the company, a project or agents are doing now, it states a UTC clock time, such as "As of 15:26 UTC". It uses the latest message's timestamp unless a record it read gives a later time. - A regression test checks that the directive contains this guidance. ## Verification - `cd server && npx vitest run src/__tests__/agent-conversations.test.ts src/__tests__/heartbeat-list.test.ts` passes (includes the prompt-size limits). - The chat-catalog eval `activity.company-overview` now grades a real observation time (stacked eval PR on `chat-eval/master-candidate`). This fix is cherry-picked onto that eval branch for the next live run. No live run has used it yet. ## Risks - Low risk. This is one guidance sentence. Overviews get one extra clause with a time. - The time is the message time, not a server clock read. Reads happen seconds after the message, so the difference is small. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:56:38 -06:00
DottaandClaude Opus 5.5	12523fb677	fix(chat): send a task whose automatic recovery stopped to the person's Retry (#20306 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A person asks the chat agent (the controller) to act on tasks that other agents own > - When a task's automatic recovery stops, for example after three runner losses, the task is blocked and the person gets a Retry control on it. Retry rechecks the task's gates and keeps the failed attempt's evidence > - The blocker told every reader "Try again or send a new message to continue". A controller read that as its own instruction. It found that Retry is board-only, then resumed the task with a comment and reassigned the task to itself > - That skips the assignee's budget and hold gates and hides the failed-run Retry from the person > - This pull request names Retry as the person's control in the blocker text, and tells the controller to point the person to it > - The benefit is that "try that task again" leads to the supported, gated Retry instead of an agent takeover ## Linked Issues or Issue Description Refs #20270 (automatic recovery after a lost local runner) What happened? In the chat eval `chat-features.runner-opencode.local.recovery.retry-exhausted`, a person asked the CEO agent to "Try the website task again" after the website task's native recovery was exhausted. The agent read the task's `executionBlocker` (`canRetry: true`, nextAction "... Try again or send a new message to continue once the previous execution has stopped."). It found that `POST /api/issues/{id}/retry` is board-only. It then posted a `resume: true` comment, which moved the task from blocked to todo, and reassigned the task to itself to re-run it. The task lost its failed-run Retry control, and the original assignee's gates were not checked. Expected behavior The agent links the original task, says why it stopped, and tells the person to use Retry there. It takes the work over only when the person explicitly asks for that. Steps to reproduce 1. Let a native run exhaust its automatic recovery (for example, three runner losses). 2. In the controller chat, ask it to try that task again. 3. The agent comments to resume the task, or reassigns it, instead of pointing to Retry. ## What Changed - `execution-blocker.ts`: the text that is added when a person can retry is now `EXECUTION_BLOCKER_PERSON_RETRY_ACTION`: "A person can use Retry on this task, or send it a new message, to continue once the previous execution has stopped. Agents cannot retry it for them." - `agent-conversations.ts`: a new chat guidance paragraph. When another task's automatic recovery has stopped, the supported next step is the person's Retry on that task. The controller links the task and explains why it stopped. It does not reassign the task, change its status, comment on it to resume it, or create a replacement task, unless the person explicitly asks it to take the work over. ## Verification - `board-access-denial.test.ts`: the directive contains the new rule. - `execution-blocker-read.test.ts`: a retryable blocker's nextAction ends with the person-Retry text. - 54 tests pass locally for these two files. - Paid live run of `recovery.retry-exhausted` with this change and #20270 on the eval branch: the run ID will be added below. ## Risks - Low. Only text changes. Any UI that shows the blocker's nextAction shows the new sentence instead of "Try again or send a new message ...". - No schema or saved-state change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:53:08 -06:00
DottaandClaude Opus 5.5	90404d2033	perf(run-logs): read object-store log tails in parallel, once, and cached (#20284 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances keep run logs in object storage as a journal of content-addressed segments and binary index nodes > - The task chat reads the last 32 KB of a run's log, aligned to a record, to show what the agent did > - A run that flushes often has many small segments. The tail read fetched each object one after another, and record alignment walked the index a second time > - On staging, one tail read took 11–12 s, and the task chat waited for it (see #20277) > - This pull request reads the tail in parallel, in one pass, and caches the objects that cannot change > - The benefit is that the first tail read is about 6.6x faster, and repeated reads (live polling, reopening a task) use only the mutable head ## Linked Issues or Issue Description What happened? `GET /api/heartbeat-runs/:runId/log?offset=…&limitBytes=32000&alignStartToRecord=true` took 11–12 s on staging for a 400 KB object-store log. Without alignment it took 6.1 s. A read from offset 0 took 2.0 s. The task chat waited for this read before it showed the run's output. Expected behavior A 32 KB tail read returns in well under a second, and a repeated read of the same tail costs about one storage round trip. Steps to reproduce 1. Use object-backed run logs (`s3_segments`). 2. Write a run log in many small flushes, as a live run does. 3. Read its last 32 KB with `alignStartToRecord=true`, and count the object GETs and how many run at the same time. Paperclip version or commit `master` at `6459488984`. Deployment mode Hosted (authenticated) deployment with S3-compatible run-log storage. Refs #20276 (measurement and plan), #20277 (chat no longer waits on a slow transcript). ## What Changed - `object-byte-journal.ts`, `readRange`: - Visits the index tree and fetches sibling objects concurrently, with at most 4 fetches for each read. - Reassembles the pieces by offset. - Cancels the remaining fetches when one fails. - The shared `objectIO` limiter still caps the process at 8 slots, and background persistence still gets freed slots first (#19176). - `object-byte-journal.ts`, `load`: - Keeps segments and index nodes in a process-wide LRU, limited to 32 MB in total and 1 MB per object. The budget counts each entry's key and a fixed 256-byte overhead with its bytes. - These objects cannot change: each key ends in the SHA-256 of its bytes, and `load` verifies that digest before it caches the object. - The mutable head is never cached. - The cache and the shared fetches are scoped to the storage provider instance. Production uses one provider in the run-log store singleton. - Concurrent fetches of the same object are shared. A shared fetch has its own controller. A reader that aborts only stops its own wait. The fetch is aborted only when no reader still waits for it, so the other readers never start a second GET for that object. - `object-run-log-store.ts`, `read`: - With `alignStartToRecord`, the first 4 KB alignment window is read in the same range as the body. - The existing 32 KB backward windows still handle longer records. - The output is unchanged. - New test `object-run-log-store-tail-read.test.ts`, written against a fake provider that counts GETs: - correct aligned and unaligned content - concurrent fetches - alignment costs at most 30 extra GETs, with no second walk of the index - a repeated read costs 1 GET (the head) - appended output is visible after cached reads - a reader that shares a fetch survives another reader's abort - a reader that joined a shared fetch stops at once on its own abort - when the reader that started a fetch aborts, no object is fetched twice - a shared fetch stops when its last reader aborts - one store's cached objects are not served to a reader of another store - a tail that starts inside a record longer than the 4 KB lookback (6 KB and 90 KB records) - the cache budget counts per-entry overhead - the shared-fetch tests hold object GETs open with a gate instead of wall-clock delays ## Verification - `cd server && npx vitest run src/services/object-run-log-store-tail-read.test.ts src/services/object-run-log-store.test.ts src/services/object-byte-journal.test.ts src/services/run-log-store.test.ts`: 4 files, 74 tests pass. The tail-read file passed 5 of 5 repeated runs. - The 18 server test files that use the journal, the run-log store or `readLog`: 381 tests pass. Two skill-materialization tests timed out under load and pass when run alone (4/4). - Red checks: - On `master`, the aligned tail read made 357 GETs with at most 1 running at a time, and a repeated read made 356 GETs. - When I removed the abort handling for shared fetches, the shared-abort test failed 3 times out of 3. - The tests for the review fixes (joiner abort, no duplicate GET after the first reader aborts, last-reader abort, per-store cache scope, cache entry overhead) fail without the fix. - An off-by-one in the long-record lookback arithmetic makes both long-record tests fail. - Benchmark: same fixture, with 30 ms latency added to each GET, a 32 KB tail of 1,200 single-record segments. \| Read \| `master` \| This PR \| \|---\|---\|---\| \| Aligned \| 11,753 ms, 365 GETs, 1 at a time \| 1,778 ms, 205 GETs, 4 at a time \| \| Unaligned \| 5,979 ms, 184 GETs \| 1,607 ms, 184 GETs \| \| Repeat aligned \| 11,686 ms, 365 GETs \| 34 ms, 1 GET \| - `pnpm typecheck` in `server` passes. ## Risks - Memory: the cache can hold up to 32 MB of decoded log bytes in each server process. Authorization still happens in the routes before any read. The cache key is the full object key, which includes the run's journal prefix. - Storage concurrency: one read now holds up to 4 I/O slots instead of 1. Background persistence still gets freed slots first. Under many simultaneous cold reads, interactive reads can reach the 32-waiter limit sooner and receive the existing retryable 503 `object_journal_io_busy`. The cache makes repeated reads cheap, which offsets this. - Remaining cost: the first read of a cold tail still walks a binary index over small segments, about 1.8 s at 30 ms per GET. A larger gain needs writer-side changes, such as wider index nodes or compaction when the log closes. Those change the persisted shape, so they are not in this PR. - No migration, configuration change, persisted-shape change or new dependency. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, browser automation, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:26:36 -06:00
DottaandClaude Opus 5.5	099adb2728	fix(recovery): settle a remote workspace save that stopped without finishing (#20291 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs in remote sandboxes save the workspace as snapshots. Each save is a `workspace_operations` row with phase `remote_workspace_snapshot` > - The ownership registry lists that kind as in-progress work, but it had no settle step > - When a restart cut off a failing final save, the row stayed `running` after its invocations had all stopped and its run had ended > - The retained-source fallback refuses while any workspace operation is running, so a task whose saved files could not be verified could never continue from its last save > - This pull request lets the ownership reconciler settle that row as failed > - The benefit is that the automatic fallback can continue such a task without any user action ## Linked Issues or Issue Description What happened? A native run in a remote sandbox finished its turn. Its final workspace save failed because the sandbox disk was full ("No space left on device" in the snapshot copy). A graceful restart then interrupted the retry. The run ended as `native_workspace_sync_out_retry_exhausted`. Its `remote_workspace_snapshot` row kept `status: running` for six days, with `saveWork: { activeIds: [], unknown: false, quiescedAt: <time> }` and `exportCleanup: { state: "pending" }`. The retained-source fallback checked the task and refused with `workspace_operation_running`, so the task stayed blocked. Expected behavior A save with no active or unknown invocation, whose run is terminal, is finished work. The reconciler must mark it failed, so the fallback can continue from the last committed save. A save that might still be running must stay untouched. Steps to reproduce 1. Run a native task in a remote sandbox. Make its final save fail, then restart the server during the retry, so the save's work is quiesced and the operation stays `running`. 2. Let the task's recovery refuse with an unverifiable-source failure (for example `allocation_ownership_ambiguous`). 3. The retained-source fallback refuses with `workspace_operation_running` on every check, and the task never continues. Paperclip version or commit `master` at `f7c85bca18`. Refs #20282 (the other cause of the same stuck task: a false ownership refusal). ## What Changed - `settleAbandonedRemoteSnapshotSave` (`recovery/workspace-finalize-attempt-settlement.ts`) settles a `remote_workspace_snapshot` row in `running` or `pending` only when all of these are true: - its save work is quiesced (`activeIds` is empty and `unknown` is false); - its run is terminal. It sets `status: failed`, records `abandonedSave` provenance and `lastFailure: remote_workspace_save_abandoned`, keeps `saveWork` and `exportCleanup`, writes a `workspace_operation.reconciled` activity entry, and wakes the workspace's terminal work. - The ownership registry wires it as the settle of `remote_snapshot_save`. - In the same transaction it re-arms blocked fallbacks. For every open hold whose fallback check last refused with `workspace_operation_running` and whose source run used this save's workspace, it clears the check and wakes the workspace-recovery queue. A hold that had already spent its 12 checks (no next check) continues as well. - The settle uses the save path's own `readSaveWork`, now exported, so it never settles save work that the save path would treat as invalid. - Because the row now has a `lastFailure`, `retireQuiescedSaves` retires it like any failed save when a later final save commits. A later retry of the same save boundary reopens a failed row, as before. - The ownership inventory row for this kind names the new settle. ## Verification - Red before the fix, green after: `settles a remote snapshot save that stopped without finishing after its run ended, and keeps open or live ones` (`ownership-reconciliation-report.test.ts`). Before: `settled: 0`. After: only the abandoned save settles. Saves with a live run, an active invocation, or unknown save work stay `running`. - Red before the fix, green after, end to end: `continues once the reconciler settles a final save a restart left running after its run ended` (`workspace-recovery-fallback.test.ts`). The fallback first refuses with `workspace_operation_running`, and the test then spends its check budget (12 checks, no next check), so the hold drops out of discovery. After one ownership reconciliation pass, the save is `failed` and the check is cleared. The worker then resolves the hold with `retained_source_unverifiable_continued_from_last_saved`. Without the re-arm, this test fails. - The reconciler test also covers save work that is not quiesced (`quiescedAt: null`), save work that is invalid (`quiescedAt: ""`), and a `pending` save. - `workspace-recovery-fallback.test.ts` (16 tests), `ownership-reconciliation-report.test.ts`, and the remote workspace lifecycle, save-concurrency, and snapshot-retention suites pass (80 tests, 1 environment-gated file skipped). - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server` passed. ## Risks - A save in another process registers its invocation in `activeIds` in the same row-locked transaction before it runs. The settle locks the row and requires `activeIds` to be empty and `unknown` to be false, so it cannot fail a save that is still working. - A save whose process died with its invocation still in `activeIds` stays `running`. That case needs an owner-death proof, which is out of scope here. - No schema or API change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, read-only browser diagnostics). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:26:04 -06:00
DottaandClaude Opus 5.5	ef2b8d5324	fix(issues): make every Retry start work or say why (#20283 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a task's run fails, the board recovers it with a Retry button, so that button must either start work or say why it did not > - An audit of every Retry-type button found three gaps. The task Retry shown during a scheduled retry's countdown returned "accepted" but changed nothing. A stale Retry looked like success in the Inbox. "Retry now" skipped the task's read and change checks, so a member who cannot see a private task could start its retry > - This pull request makes the task Retry start a scheduled retry immediately, makes every Retry that did not start work say why, and gives "Retry now" the same authorization as the task Retry > - The benefit is that no Retry click silently does nothing ## Linked Issues or Issue Description Refs #19831. The board reported a dead Retry button. A read-only audit of every Retry button confirmed: - Task Retry during a scheduled retry was a no-op. `TaskInterruptionPresentation.canRetry` is true in `retry_scheduled`, so `ExecutionBlockerNotice` shows Retry. `requestTaskRetry` treats `scheduled_retry` as busy and inserts a `coalesced` wake pointing at that run. `resumeTaskRetry` reads only `deferred_issue_execution`, and dispatch uses coalesced user wakes only to carry comments. Nothing promoted the run: the API answered 202 `accepted: true` and the countdown did not change. - A stale Retry looked like success. `accepted: false, stale: true` comes back as HTTP 200. The Inbox ignored `accepted` and opened the task as if work had started. The task page showed no reason. - "Retry now" checked less than the task Retry. It ran only `assertBoard` and `getAccessibleResource`, skipping `assertIssueReadAllowed` and `issue:mutate`. A company member who cannot read a private task could promote that task's scheduled retry. - "Retry now" said nothing for some outcomes. It toasted only `promoted` and `gate_suppressed`. ## What Changed - `server/src/services/task-retry.ts`: when the Retry coalesces into the task's latest run and that run is `scheduled_retry` (with no other Retry already active), the receipt returns `scheduledRetryRunId`. - `server/src/routes/issues.ts`: - `POST /issues/:id/retry` then promotes that run through the existing `retryScheduledRetryNow`, the same path "Retry now" uses. It logs `issue.scheduled_retry_retry_now` with `source: "task_retry"` and returns `scheduledRetry: { outcome, message }`. - `POST /issues/:id/scheduled-retry/retry-now` now runs `assertIssueReadAllowed` and `decideIssueAccess("issue:mutate")`, matching `/retry`. A private task the member cannot read now answers 404. - `ui/src/hooks/useTaskRetry.ts`: new `taskRetryNotStartedReason`. When a Retry did not start work (stale, or a scheduled retry that is gate-suppressed, waiting, or gone), the hook returns that reason as its error. `ExecutionBlockerNotice` and `StalledDependencyNotice` already display it. - `ui/src/pages/Inbox.tsx`: a task Retry that did not start work shows the reason in the error toast and does not navigate. - `ui/src/hooks/useRetryNowMutation.ts` and `IssueScheduledRetryCard.tsx`: every "Retry now" outcome now shows its headline and message, including `no_scheduled_retry` on the card. - `ui/src/api/issues.ts`: `retryTask`'s response type includes `scheduledRetry`. ## Verification Red/green (each new test fails against master and passes with this PR): - `server/src/__tests__/issue-scheduled-retry-routes.test.ts`: - "starts a scheduled retry now when the task Retry is clicked during its countdown": 202 with `scheduledRetry.outcome: "promoted"`, the same run becomes `queued`, and no second retry run is created. - "refuses Retry now on a private task the member cannot see, like the task Retry": on master "Retry now" answers 200 `promoted`; now both routes answer 404 and the run stays `scheduled_retry`. - The seed now adds the board user's membership, which the stricter route needs. All 33 tests pass. - `ui/src/hooks/useTaskRetry.test.tsx`: a replaced attempt, a gate-held scheduled retry and a waiting scheduled retry each show their reason; a promoted one shows nothing extra. - `ui/src/pages/Inbox.test.tsx`: an Inbox Retry that did not start work (stale, or gate-held) toasts the reason and does not navigate. - `ui/src/components/IssueScheduledRetryCard.test.tsx`: "explains when there was no scheduled retry left to start". Other Retry buttons: the chat failed-run Retry already reports stopped and failed retries (`agents.test.ts`). Agent lifecycle "Retry now" and cost "Retry accounting" start no task work and already surface their API errors. Regression and static checks: - UI: 7 files, 168 passed. Server: task-retry and scheduled-retry suites, 149 passed. `issue-agent-mutation-ownership-routes.test.ts` passed. - `maintenance-server-routes.test.ts` "assembled maintenance server production wiring" fails identically on master on this Mac (`ensure-build-deps`). - `tsc --noEmit` passes for `server` and `ui/src`. `pnpm check:token-gates` is clean. ## Risks - A task Retry during a scheduled retry now starts the run immediately, through the existing "Retry now" promotion gates (pause, budget, cleanup waits). A gate that holds it is reported, not bypassed. - "Retry now" is stricter: a member without read or change access to the task gets 404 or 403, as on `/retry`. - No schema change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:25:21 -06:00
Dotta	5c73633a8f	fix(workspaces): preserve remote recovery holds during online import (#20298 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Online workspace import preserves old controller-local files while unrelated remote work continues. > - A task can have an old local checkout and a separate durable remote workspace. > - A terminal remote run can retain a file-recovery hold on that task. > - The legacy source checker rejects remote references, so that hold blocks independent local imports. > - This pull request proves that the hold belongs only to the separate remote source and fences its finalizer during import. > - Old files can migrate while the remote recovery obligation stays intact. ## Linked Issues or Issue Description Refs #19873 and #19854 for the existing cohort and source-verification paths. Searched related online migration and remote recovery PRs; no duplicate fix was found. What happened? Online import rejects a legacy Git checkout with `online_workspace_source_scope_unproven` when a same-task failed run has a prepared reference to a separate durable remote workspace. The legacy run reader rejects remote references before the hold can be classified. Expected behavior Independent local files can import under their source fences. The separate remote workspace, dirty writer, allocation, accepted result and recovery hold must stay intact. Steps to reproduce 1. Create related local Git workspaces. 2. Record a terminal failed native run and an exhausted file-recovery hold for a separate durable remote workspace on the same task. 3. Import the local leaf online. The old checker refuses before capture. Paperclip version or commit Reproduced on master at `052c638b519ebb15f81e4fcd1a96adf6cd3b2f10`. Deployment mode Managed server with PostgreSQL and remote Runner execution. ## What Changed - Add a distinct remote-only preservation proof. Keep remote runs excluded from legacy export sources. - Use the canonical source inspector and prepared-reference matcher. Bind the run, finalizer, action, task, accepted result, authority, allocation and prior committed snapshot. - Acquire the held run's finalizer fence through the existing sorted lock path. Existing source-proof reinspection rejects changes before publication. - Preserve previous-release prepared references with an older baseline snapshot. Do not require that snapshot's epoch or source run to equal the current run. - Add real PostgreSQL import, authenticated cold restore, preservation, refusal, lock-contention and publication-race regressions. Document the behavior. ## Verification - Genuine baseline failure on unchanged product: the real importer refuses with `online_workspace_source_scope_unproven`. - Final focused regression file: 23 tests passed in 13.19 seconds through the canonical isolated PostgreSQL wrapper on Node 24.22.2. - 24 existing affected migration/recovery tests passed against the same final production bytes. Another 83 tests were explicitly excluded by the focused selection; no full-suite pass is claimed. - The positive imports local bytes, confirms the remote finalizer fence is held during upload, leaves all remote records unchanged, parks the original repository and authenticates/restores the archive. - Refusal and race cases cover malformed, foreign, same-local and mixed references; active owners/writers; changed allocation, result, task, action, authority and baseline; and finalizer contention. - Server TypeScript check, module-boundary check and `git diff --check` passed. - An intermediate run had one incorrect assertion that expected an explicit empty legacy-source array. The canonical journal omits that empty field. The final assertion accepts absence and still forbids any exported remote source. Final relevant results are 23 new plus 24 existing passes, rather than a claim that the earlier invocation passed in full. - Required CI passed on head `6a278e717db2b5f614b0a595f99bfd0fb5430227`: 55 successful checks and 4 intentional skips. [CI run](https://github.com/paperclipai/paperclip/actions/runs/42650025033). Greptile reviewed that head at 5/5 with no actionable findings or open inline/review comments. Hosted continuation remains pending normal delivery. ## Risks - A false disjointness proof could admit an unsafe local import. Strict company/task/source binding, the canonical inspector, finalizer fencing and publication revalidation retain refusal on uncertainty. - This does not recover or settle remote files. Dirty authority and the original recovery obligation remain unchanged. - No schema, public API, archive format or saved-run identity changes. Existing persisted state carries forward. ## Model Used OpenAI Codex, GPT-6.1 Sol (`gpt-6.1-sol`), with reasoning, repository tools and code execution. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-09 10:13:41 -06:00
DottaandClaude Opus 5.5	d10c6e0e8b	fix(chat): hold a chat message at the agent's daily limit instead of dropping it (#20292 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat sends each person's message to the conversation agent through a durable outbox (`deliverConversationComments`). The outbox keys each wake receipt as `conversation-comment:<commentId>`. > - The outbox treats any receipt with that key as delivered. Queue admission writes a `skipped` receipt when the agent reached its daily run or cost limit. > - So a chat message sent after the agent used its daily allowance was consumed silently. No run started, the chat showed nothing, and nothing resumed it. > - A person's message must never be silently dropped. The daily limit is an operator control, so the message must wait honestly, not bypass the limit. > - This pull request makes the outbox hold the message at the daily limit, like it already does for a paused agent, and resume it when the limit resets. > - The benefit is that a chat message at the daily limit is saved, shown with its reason, and sent automatically at 00:00 UTC. ## Linked Issues or Issue Description No public issue exists. Inline bug report: What happened In a chat eval, the conversation agent (CEO) had `maxDailyRuns: 3`. After three chat turns and a separate failed-model phase on another agent, the person sent a fourth message. The comment was saved (POST 201), but no CEO run started. The conversation stayed `active` with no `executionRunId`. The ownership reconciler later reported `I7_visible_next_action`. The server log shows no wake error. Queue admission had written a `skipped` receipt (`heartbeat.daily_run_limit`) under the outbox key, so the outbox counted the message as delivered. Expected behavior A chat message that reaches the agent's daily limit is not consumed. It stays saved, the chat shows why it waits, and it is sent automatically when the limit resets. The daily limit still applies: the message does not start a run early. Steps to reproduce 1. Set a conversation agent's `runtimeConfig.heartbeat.maxDailyRuns` to N. 2. Send N chat messages and let each turn finish. 3. Send one more chat message. 4. See that no run starts, the chat shows no waiting state, and the message is never answered (a `skipped` wake receipt with reason `heartbeat.daily_run_limit` exists for `conversation-comment:<id>`). ## What Changed - `server/src/services/agent-conversations.ts`: add `conversationDeliveryHold`, which applies the same daily run/cost limit check as queue admission. The outbox returns `deferred` at the limit and writes no receipt, so the message stays owed. It reports the next UTC day start through a new optional `onDeferredDeadline` option. - `server/src/services/recovery/service.ts`: stranded-work recovery passes its `onDeferredDeadline` to the outbox, so the held message is retried when the limit resets. - `server/src/services/heartbeat/queue.ts`: if the limit is reached while an outbox wake is in admission (a race), admission throws a 409 with `reason: heartbeat.daily_*` for `conversation-comment:` keys instead of writing a `skipped` receipt. The outbox already maps a matching 409 to `deferred`. Other wakes keep the current skip behavior. - `server/src/routes/issues.ts` (`GET /issues/:id/queued-comments`): when a chat has no queue receipt but has messages held by the daily limit, return them as a `deferred` queue with `executionWait` (reason and message). Edit, send and discard stay off because the outbox owns delivery. - `ui/src/components/task-chat/TaskChatQueuedMessages.tsx`: show the `executionWait` message for such an outbox-held queue (no queue id and no target run). Receipt-backed queues keep the current behavior and do not duplicate the task status. - Tests: three real-database tests in `agent-conversation-delivery.test.ts` (hold and resume after the limit resets, recovery deadline, and a limit reached during real `heartbeatService.wakeup` admission) and one UI test. ## Verification - `cd server && pnpm exec vitest run src/__tests__/agent-conversation-delivery.test.ts`: 10 passed. Without the source change, the three new tests fail (`expected 'delivered' to be 'deferred'`; the real queue path writes the skipped receipt). - `cd server && pnpm exec vitest run src/__tests__/issue-recovery-conversations.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/heartbeat-undispatched-execution-wait.test.ts src/services/heartbeat/queue.test.ts src/__tests__/agent-chat-messages.test.ts`: 215 passed. - `cd ui && pnpm exec vitest run src/components/task-chat/TaskChatQueuedMessages.test.tsx`: 36 passed. - `pnpm exec tsc --noEmit -p server/tsconfig.json`: clean. `node scripts/check-token-gates.mjs`: clean. - Manual: give a chat agent `maxDailyRuns: 1`, send two messages. The second message shows in the queued messages bar with "<agent> reached its daily run limit. Your message is saved and sends automatically when the limit resets at 00:00 UTC." ## Risks - Behavior change for chat only: a chat message at the daily limit now waits in the outbox instead of being consumed. It is sent after 00:00 UTC. If an operator raises the limit, the message is sent on the next recovery pass for that chat; the UI message does not promise an earlier send. - Saved state from the previous release: no persisted shape changes. Messages that an earlier release already consumed with a `skipped` receipt stay consumed. They are not replayed later, which avoids sending old messages without notice. - The queued-comments read adds one agent read and one daily-limit count for a chat with no queue receipt. This is the same query that admission runs. - The recovery deadline is in-process; instance-sleep scheduling does not get a new deadline from this change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 10:03:28 -06:00
DottaandClaude Opus 5.5	87e755ce8e	fix(recovery): verify cross-task warm sandbox handoffs and say plainly when recovery refuses (#20282 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs in remote sandboxes keep a warm sandbox and hand it to the next run through a chain of reusable leases > - Acquisition binds that chain to the execution workspace, so a task can inherit a warm sandbox from an earlier task in the same workspace > - Saved-files recovery verifies the same chain, but it required every earlier lease, run, and finalization to belong to the recovering task > - A valid chain that started with another task therefore always failed with `allocation_ownership_ambiguous`. That code is `service_required`, so recovery stopped after one attempt, and each board Retry repeated the same refusal > - This pull request makes the chain check match the acquisition rule, and it checks each predecessor against its own task > - It also makes the board say plainly when recovery refuses the saved state, because "failed after automatic retries" was not true and Retry looked like it did nothing > - The benefit is that a task blocked with "Saving the agent's files failed after automatic retries" can recover its saved files, and a refusal that Retry cannot change is shown with its reason ## Linked Issues or Issue Description What happened? A native run in a remote sandbox finished its turn, and its result was accepted. The copy of its files back to the host failed and was scheduled to retry. A graceful server restart then interrupted the retry, so the run settled with `native_workspace_sync_out_retry_exhausted`, and the task became blocked. Every saved-files recovery generation then failed on its first attempt with `allocation_ownership_ambiguous` (`service_required`). The board showed "Saving the agent's files failed after automatic retries. Retry to try again". Each Retry replaced the generation (`issue.recovery_generation_replaced`), and the new generation failed in the same way within one second. To the user, Retry did nothing. The sandbox lease chain was valid: same agent, same execution workspace, same sandbox, and matching handoff receipts on all eight leases. But its first three leases belonged to an earlier task in the same workspace. Expected behavior Recovery must accept a warm sandbox chain that acquisition accepted. Acquisition binds reuse to the execution workspace, and it requires the same task only when the lease has no workspace. Recovery must check each predecessor run and finalization against that predecessor's own task. It must still refuse a chain whose records disagree or that has no shared workspace. Steps to reproduce 1. Run a native task in a remote sandbox with `reuse_by_environment`, with an execution workspace that another task also uses. 2. Run the other task in the same workspace first, so the sandbox lease is handed from that task to this one. 3. Make the last run of the second task fail to copy its files back (for example, restart the server during the copy-back retry). 4. Wait for saved-files recovery, or click Retry. Recovery fails with `allocation_ownership_ambiguous`, and Retry repeats the failure. Paperclip version or commit `master` at `a3a25ce6c9` (the same code as the 2026.1106.0 staging build for every file this change touches). ## What Changed - `retiredReusableLeasePredecessors` (`environment-lease-handoff.ts`) requires the same task only when the lease has no execution workspace. This matches the binding that `acquireLease` uses when it replaces a reusable lease. - `inspectWorkspaceRecoverySource` (`workspace-recovery-source.ts`) checks each predecessor run and its finalization against the predecessor lease's own task, not the recovering task. A predecessor from another task is accepted only when both leases have the same execution workspace. All other checks stay: same agent, terminal run, no live controller or lease owner, a settled finalization, and a saved snapshot at an older epoch on the same allocation, workspace, and remote directory. - The other callers of the chain walk (remote termination resolution and retired backup restore) get the same rule. - The test fixture `warmHandoffSource` gets an option that starts the chain with an earlier task in a shared workspace. - Honest copy for exhausted recovery (`exhaustedSystemRecoveryAction`): - A deterministic refusal (`service_required`) now says "Paperclip refused to recover the saved files", names the reason in plain words and the code, and says that Retry runs the same check again and succeeds only after Paperclip is updated. - Other exhausted recovery says "stopped after N attempts" only when there were several attempts. One attempt says "failed". - Continuation (`explicit-native-continuation.ts`) reports a new wait reason, `saved_work_recovery_refused`, when the exhausted recovery is a deterministic refusal. The task presentation shows its own message for it. `saved_work_recovery_failed` no longer claims automatic retries. - Retry stays available for a refusal. It is the action that runs recovery again after an update. After each Retry, the board shows the same refusal and its reason, not an unchanged generic notice. ### Recovery refusal classes \| Retry class \| Failure codes \| Automatic recovery \| Board Retry \| \| --- \| --- \| --- \| --- \| \| `retry_transient` \| `provider_unavailable`, `export_interrupted` and other transient codes \| Retries with backoff, up to 5 retries in 30 minutes \| Starts a new recovery round \| \| `recheck_evidence` \| `agent_stop_unconfirmed`, `workspace_writer_active`, `effect_unresolved`, `copyback_stop_unconfirmed` \| Checks the evidence again later \| Starts a new round (the copy-back stop case has its own wait text) \| \| `use_alternative_strategy` \| `historical_source_missing`, `source_unavailable` (no recorded source) \| Stops. The retained-source fallback can continue from the last save \| Starts a fresh attempt \| \| `service_required` (deterministic) \| `allocation_ownership_ambiguous`, `source_binding_invalid`, `snapshot_verification_failed`, `archive_unsafe`, `source_unavailable` (recorded source gone) \| Stops after one attempt. The same saved state gives the same result \| Runs the same check again. The board now says this and names the reason \| ## Verification - Red before the fix: `recovers saved work after 1\|2 warm handoffs that began with another task in the same workspace` failed with `expected 'allocation_ownership_ambiguous' to be null`. - Green after the fix: `npx vitest run src/services/workspace-recovery-warm-handoff.test.ts src/services/workspace-recovery-source.test.ts` passed (60 tests). - Safety cases that stay fenced: a predecessor lease that names a different task than its run (`lease_task`), and a cross-task predecessor with no shared workspace (`unshared_workspace`). The existing `task` fault (another task with no workspace) also stays fenced. - Sibling and retry suites passed: environment runtime and orphan sandbox, cancelled workspace preservation, workspace recovery fallback, task retry, remote workspace lifecycle, explicit native continuation, native cleanup admission, and remote retry admission. - Copy, red before and green after: `system-recovery-state.test.ts` (`states a deterministic refusal plainly and never claims retries that did not happen`), and the end-to-end `names a deterministic saved-files refusal behind Retry when no fallback can continue, and another Retry shows it again` in `workspace-recovery-preparation-gates.test.ts`. On the old code the wait reason was `saved_work_recovery_failed`. With the change it is `saved_work_recovery_refused`, with the refusal reason, for both Retry clicks. - Updated expectation: the board handoff in `workspace-recovery-retry-copyback.test.ts` now expects the refusal text for its `source_binding_invalid` source (79 tests pass). - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server` passed. - I compared the real incident's lease chain with every predecessor condition through read-only diagnostics. All eight leases match on agent, sandbox, workspace, and remote directory, and each predecessor run saved a snapshot at an older epoch. ## Risks - Recovery now trusts a predecessor from another task in the same execution workspace. Acquisition already allows that handoff, and recovery still requires the predecessor's own run and finalization to be settled and saved. A workspace is the unit that owns the files, so this does not let one workspace read another workspace's files. - I could not read one predecessor's finalization row in the real incident. If that row is not in a settled phase, recovery still refuses with `allocation_ownership_ambiguous`, which is the old behavior. - `snapshot_verification_failed` is labeled `service_required` at its 19 call sites, so it now shows the refusal text. If one of those sites is really transient, the text says "Retry runs the check again", which is still true. - A second cause kept the same task stuck: a remote snapshot save that a restart interrupted stayed `running`, and that blocks the retained-source fallback. That fix comes in a separate pull request. - No schema or API change. The new wait reason is a new string in an existing field. The UI shows the message as it receives it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, read-only browser diagnostics). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:53:37 -06:00
Nicky LeachandPaperclip	052c638b51	fix(environments): honor stopped-task proof after warm preparation (#20289 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A warm plugin Runner retains its task for later runs. > - A follow-up can find that the retained task has stopped. > - The resume path already asks settle to check status before it sends stop. > - A saved startup marker disabled this order, even after warm preparation had finished. > - This pull request applies the same stop-proof check to all observed-stop releases. > - The next run skips a repeated stop when status proves both execution and required preparation have stopped. ## Linked Issues or Issue Description Fixes: #20224 Refs #20203 ## What Changed - Remove the startup-marker exclusion from the existing status-first path. Keep the existing shared confirmation check. It still requires `preparationStopped: true` for startup work. - Keep stop followed by status when the first status lacks the required proof. Keep the startup drain, credential writeback, complete, and cleanup failure handling. - Add retained-task coverage with the startup metadata that warm preparation saves. Cover a startup cleanup whose first status lacks preparation proof. - Update the plugin task lifecycle contract. ## Verification - Local macOS, source base `3e8744fa91`: from `server`, `pnpm exec vitest run src/__tests__/environment-task-runtime.test.ts src/__tests__/plugin-warm-tasks.integration.test.ts src/__tests__/plugin-task-credential-writeback.test.ts` passed: 3 files, 82 tests. The integration tests use real embedded PostgreSQL and controlled plugin results. - After restoring the fix, the two affected files passed again: 73 tests. - Independent parent verification on commit `19f720ff68`: the two files filtered to the changed startup-proof and retained-task cases passed (4 tests). Independent source review found no actionable issue. - Baseline proof: with only the source condition restored to upstream, the new retained-task test with saved startup metadata fails because it sees an extra `stop`. The case without that metadata passes. The source fix was then restored. - The retained-task regression asserts exactly `environmentResumeLease`, `status`, `complete`, and `environmentAcquireLease`, with both task operations bound to the old task. The startup test also proves that missing preparation proof sends `stop` and a second `status`. - Test setup: `node scripts/agent-preflight.mjs --skip-rust-debug --explain` installed dependencies, then stopped because Cargo is unavailable on this Mac. Initial test collection failed because the plugin SDK output was absent. `pnpm --filter @paperclipai/plugin-sdk build` built that required dependency and the shared package; the targeted test run then passed. - Standard Greptile reviewed commit `19f720ff686058b4dd6b34f27cdfb9c8ff60e50a` at 5/5 with no review threads. [Review](https://github.com/paperclipai/paperclip/pull/20289#issuecomment-6743829614). - Full local checks were not run, as requested. [GitHub PR workflow 42644023868](https://github.com/paperclipai/paperclip/actions/runs/42644023868) passed on final commit `19f720ff686058b4dd6b34f27cdfb9c8ff60e50a`. It supplied broad typecheck, build, and test coverage. The separate Snyk status passed. Optional Storybook checks were skipped. - The change removes one provider stop call on the reported path. The issue's measured eight calls would become seven if other calls stay the same. The earlier five-call target is not claimed. No deployed latency measurement is claimed. ## Risks - A provider that cannot prove preparation stopped still gets stop and status. That fallback can cost one extra status call, as the existing status-first contract already allows. - A status failure keeps cleanup pending and blocks replacement. No termination proof or credential handoff is skipped. - No schema, API, dependency, or timer change. No added work on successful task reuse or ordinary task dispatch. - Live provider acceptance remains unrun. The local evidence checks orchestration and stop-proof handling. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, repository inspection, shell tools, and code execution. The exact serving model ID and context window are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-09 07:44:19 -08:00
DottaandClaude Opus 5.5	dc9ad17515	perf(storage): time workspace snapshot save and restore steps and count storage retries (#20280 ) ## Thinking Path - Cloud durable storage is a major source of failures and slowness on staging. For example, a staging task failed when saving files after automatic retries. - Before measuring or fixing it, we need to see where storage time goes. Today a saved-workspace restore before the turn appears only inside the coarse `workspace_environment_realize` phase. - After the turn, the save's encrypt, put, verify and multipart throttle backoff have no timing at all. Only `captureMs`, `transferMs`, `verifyMs`, `uploadMs` and `publishMs` exist, in `saveTiming`. - Storage retries happen in the SDK and in the multipart uploader, invisibly. - This adds the measurement first, so the storage-mode chaos waves (MinIO behind a fault proxy) can report save and restore p50/p95 and point at the dominant wait. ## Linked Issues or Issue Description What happened? Durable-storage saves and restores were opaque in the run log and save timing. A slow or repeatedly retried storage call could not be told apart from slow capture, encryption or sandbox work. Steps to reproduce 1. Run a remote task with S3-compatible durable storage, so it saves after the turn and restores on the next run. 2. Inspect the run log and the `remote_workspace_snapshot` operation's `saveTiming`. The restore's download, decrypt and verify, and the save's put, verify and throttle backoff, have no timing. Expected behavior Each storage step has a duration, and storage request retries and backoff are counted. ## What Changed - `server/src/storage/request-stats.ts` (new): an async-local collector of storage request statistics. Counts only: - `requests`, `attempts` and `failedRequests`; - `retryDelayMs` (SDK waits); - `parts`, `throttled` and `backoffMs` (the multipart uploader's waits). - `server/src/storage/s3-failure.ts`: `withS3FailureDiagnostics`, which wraps every S3 send, records each request's SDK `$metadata.attempts` and `totalRetryDelay`, on success and on failure. - `server/src/storage/s3-multipart.ts`: records each uploaded part and each resend wait, flagged throttled when the backend returned 503, 429 or SlowDown. - `server/src/services/remote-workspace-snapshot-storage.ts`: - Each step is a `storage.snapshot.` run phase: - save: `prior_attempts`, `encrypt`, `digest`, `put`, `verify`; - restore: `download`, `decrypt`, `verify_plaintext`, `publish`. - A caller can pass a `timing` record that receives the step durations, the bytes and the request statistics. - `server/src/services/remote-workspaces.ts`: - The save passes a timing record, kept in `saveTiming.storage`. - The sandbox unpack during a restore is its own phase, `workspace.snapshot.sandbox_restore`. - `execution-preparation-timing.ts`: the new phase names. - `doc/run-log-events.md`: a new "Durable storage timing" section. Because the restore runs inside run preparation, its steps of 50 ms or more become `run.performance.span` rows with no extra plumbing. This is run-log and instance-database data only; there's no Telemetry change. ## Verification - `server`: `storage-s3-multipart-backoff`, `storage-s3-multipart`, `storage-s3-multipart-throttle`, `storage-s3-failure-diagnostics`, `remote-workspace-snapshot-storage` and `execution-preparation-timing` tests: 86 passed. They include: - collected stats for a throttled multipart upload: 12 requests, 7 failed, 7 throttled, 35.5 s backoff and 3 parts; - a save and restore that fill every step duration and the byte count. - `remote-workspaces.integration.test.ts`: 112 passed. Its `saveTiming` assertion now includes `storage`. - Server typecheck: no errors in the touched files. ## Risks - Overhead:* one async-local lookup per S3 request, plus timestamps per snapshot step. - Collector scope: the collector only exists inside a snapshot save or restore. Other storage callers are unaffected; they record nothing. - `saveTiming` gains a `storage` object. Readers that compare it exactly (one test, updated) will see the new field. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`, Anthropic), 1M context window, tool use and code execution, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:41:53 -06:00
DottaandClaude Opus 5.5	3ef45fdd92	fix(recovery): let a new task wait for its manually paused assignee (#20281 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A person can pause an agent. Tasks assigned to a paused agent wait until the person resumes it. > - Stranded-task recovery looks for assigned work that cannot continue and moves it to `blocked` for the board. > - A task created for a manually paused agent has no run yet. Recovery saw an assignee that is not invokable and moved the new task to `blocked` a few seconds after creation. > - The person paused the agent on purpose. The blocked task and its "the board must choose the next action" comment fight that decision, and the person must unblock the task by hand after resume. > - This pull request lets new `todo` work with no run wait while a manual pause lasts. The existing assigned-todo dispatch starts it when the agent is resumed. > - The benefit is that work queued for a paused agent starts on resume, as the pause promises. ## Linked Issues or Issue Description What happened A chat evaluation pauses an agent (Morgan) and then runs `paperclipai issue create ... --assignee-agent-id <Morgan> --status todo`. The CLI returns the task with status `todo`. Within about five seconds the task changes to `blocked`, with the system comment "Paperclip cannot safely continue automatic recovery because the original assignee is not invokable. The source assignment is unchanged and the board must choose the next action." The server log shows only the expected 409 "Agent is not invokable" for the assignment wake. Expected behavior The task stays `todo` while the person keeps the agent paused. When the person resumes the agent, the task starts once. A comment and a `blocked` status are not added. Steps to reproduce 1. Pause an agent from its page. 2. Create a `todo` task assigned to that agent. 3. Wait for the stranded-task recovery sweep. See that the task is `blocked` with the comment above. ## What Changed - `server/src/services/recovery/service.ts`: in `reconcileStrandedAssignedIssues`, skip a `todo` task with no run when its assignee is paused with a `manual` lifecycle hold. The existing rule above it already skips work whose run the pause cancelled. Other pause causes, such as a budget hold, keep the existing escalation. - `server/src/__tests__/heartbeat-paused-assignee-new-task.test.ts`: a PostgreSQL test. The task stays `todo` with no comment and no run during two recovery sweeps. After resume it starts exactly once and finishes. A budget-held assignee still escalates. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-paused-assignee-new-task.test.ts`: 2 passed. Without the fix, the first test fails with `expected 'blocked' to be 'todo'`. - `npx vitest run src/__tests__/issue-recovery-actions.test.ts src/__tests__/heartbeat-refused-assignment-wake.test.ts`: passed. - `src/__tests__/issue-tree-pause-resume-run.test.ts` had timing failures on this low-memory machine both with and without this change (different tests each time). Those tests do not pause agents, so this change does not reach them. CI runs them on this branch. ## Risks - Low risk. The skip needs all of these: the agent is paused, it has a `manual` hold, the task is `todo`, and the task has no run. Work that already ran keeps the current rules. - No persisted state changes shape. State saved by the previous release carries forward: a task that the previous release already moved to `blocked` stays `blocked`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #20143 (a pause's stop never blocks the task behind an incident), #9998 (open: reject paused agents as assignment targets). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:26:20 -06:00
DottaandClaude Opus 5.5	6db0bb9776	fix(chat): tell agents which CLI commands a person runs in their own terminal (#20267 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People talk to agents in chat. The chat guidance in `server/src/services/agent-conversations.ts` tells every chat agent how to answer. > - People often ask a chat agent for a terminal command: create a task, download a plan, inspect or stop a run, or point the CLI at a company. > - The guidance did not name the Paperclip CLI. Agents guessed: they did the work themselves, wrote `curl`, used placeholder company ids, or invented an API address. > - The person then has to fix the command, or gets a file in the agent's workspace instead of on their own machine. > - This pull request adds one guidance paragraph. It names the supported `npx paperclipai` commands, asks for real ids from the conversation, and asks for the person's own Paperclip URL as a placeholder. > - The benefit is a command the person can run as it is, with the right company and the right actor. ## Linked Issues or Issue Description What happened Three chat evaluations (cli-operations) asked for terminal commands. "Show me how to point my terminal at the Acorn company." got `context set` with `<acorn-company-id>` and an invented `http://localhost:3100`, although the agent can read its company id. "Download this task's plan from the terminal as a file." got a file saved in the agent's own workspace and an attachment link, with no terminal command. "Show me the command to inspect Morgan's last run and stop it if it is still running." got `curl` with environment variables that exist only in the agent's runtime. Expected behavior The agent gives `npx paperclipai` commands that the person runs in their own terminal. The commands use real ids that the agent can read in the conversation. The Paperclip address is the placeholder `<your Paperclip URL>`. The agent says which commands need a board user. Steps to reproduce 1. Open a chat with an agent. 2. Send "Download this task's plan from the terminal as a file." 3. See that the agent saves the file itself and gives no terminal command. ## What Changed - `server/src/services/agent-conversations.ts`: add a chat guidance paragraph for terminal and CLI requests. It names the supported commands for task creation, document download, run read and cancel, and context profiles. It asks for one command per line, real ids, and the `<your Paperclip URL>` placeholder. - `cli/src/__tests__/chat-cli-guidance.test.ts`: a new test reads the guidance and resolves every `npx paperclipai` command against the registered commander commands. It fails on an unknown command, flag or missing argument. - `server/src/__tests__/agent-conversations.test.ts`: check that the paragraph is in the directive and in the chat prompt. ## Verification - `cd cli && npx vitest run src/__tests__/chat-cli-guidance.test.ts`: 3 passed. - `cd server && npx vitest run src/__tests__/agent-conversations.test.ts src/__tests__/board-access-denial.test.ts`: 80 passed. - The chat evaluations `cli-operations.choose-cli-context`, `export-document-cli` and `inspect-run-cli` run with this commit on an evaluation branch. Results are added in a comment. ## Risks - Low risk. The change adds prompt text and tests. Routes, permissions and responses do not change. - The chat prompt is about 300 words longer. - If a CLI command or flag changes, the new CLI test fails until the guidance changes too. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #20029 (terminal sign-in guidance in the API catalog). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:26:13 -06:00
Pavel-GuseynovandClaude Opus 5.5	4d297b998a	fix(server): give remote MCP tool calls a 30 second default deadline and record stream failure evidence (#19768 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The tool gateway calls remote MCP servers over Streamable HTTP, and a server can answer `tools/call` with a `text/event-stream` body > - Such a server can send progress notifications before the response, and a slow tool can take more than 10 seconds > - The gateway aborts a remote MCP `tools/call` that uses the default deadline after 10 seconds, the general tool default > - When the call fails, the call evidence does not show how far the stream got, or whether the cause was the deadline or a broken transport > - This pull request gives remote MCP calls a default deadline of 30 seconds, the default response timeout of the remote HTTP client, and records the failure cause and the stream progress > - The benefit is that slow remote tools complete, and an operator can tell a deadline from a peer reset in the call evidence ## Linked Issues or Issue Description Refs #10147. That issue reports that a named MCP gateway aborts remote tools after 10 seconds and ignores the connection timeoutMs. This pull request raises the default deadline of a remote MCP `tools/call` to 30 seconds. It does not make the named gateway read the connection timeoutMs. Related: #11910 (open) keeps a connection callable after one timed-out tools/call. On current master a timeout and a JSON-RPC response error already keep the connection healthy, and this pull request keeps that policy. I searched pull requests and issues for "event stream tools/call", "SSE MCP tool call hang", "remote MCP timeout stream", "mcp text/event-stream" and "tool_timeout remote MCP". The only related open pull request is #11910. ## What Changed - `server/src/services/remote-http-fetch.ts`: the default response timeout is exported as `DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` (30,000 ms, unchanged value). - `server/src/services/tool-gateway.ts`: - A remote MCP `tools/call` that uses the default deadline now gets `DEFAULT_REMOTE_HTTP_RESPONSE_TIMEOUT_MS` instead of the general tool default of 10 seconds. The Railway `run-command` budget and an explicit `timeoutMs` keep their values. - The call evidence records the response status, content type and request id before the body is read. A body that fails partway records the bytes received so far as its size. - A failed call records `failureKind` (`invocation_timeout`, `protocol_failure` or `transport_failure`), the deadline and whether it was aborted, the stream progress, and the error with its cause chain. Each string in the chain is redacted with `redactSensitiveText`, and a cyclic cause chain is cut with a "[Circular cause]" marker. - A deadline abort is a 504 `tool_timeout` and starts a fresh MCP session on the next call, as before. The connection health policy of master does not change. - A deliberate stop of a call (`elicitation_required`, `provider_interaction_required`, `oauth_insufficient_scope`, `oauth_refreshed_retry_required`) keeps its own reason when the deadline passes while it is recorded, as on master. An error that is already a gateway error also gets a failure kind: `tool_timeout` is `invocation_timeout`; a malformed, invalid or oversized response is `protocol_failure`; and `remote_http_*` (for example a DNS failure) and `mcp_remote_fetch_failed` are `transport_failure`. - A non-success response body is read as a stream, so a body that fails partway records the bytes that arrived. The reader stops at the size limit and cancels the body, as the `tools/call` reader does. #19785 makes the same reader change for the context path. - `server/src/services/mcp-http.ts`: `readMcpHttpResponse` takes an `onProgress` callback. It reports bytes, chunks and events received, whether the matching response arrived, and whether the stream ended. As on master, the read returns as soon as the response with the matching id arrives. - Tests: - `server/src/__tests__/mcp-http.test.ts`: a partial event followed by a peer error keeps the progress that was consumed, and releases the reader. - `server/src/__tests__/tool-gateway.test.ts`: six scenarios run against a remote server that streams one progress event first: slow success after 12 seconds, the default deadline, an explicit 1,500 ms deadline, a peer reset, a cyclic error cause, and a primitive error value. - `server/src/__tests__/tool-gateway.test.ts`: three regression tests: a tool question sent after the deadline passed keeps `elicitation_required`; a DNS failure records `transport_failure`; and a 503 body that fails partway records the bytes that arrived. All three fail on `717e6c77c` and pass on this head. - `server/src/__tests__/tool-gateway-service.test.ts`: when the call deadline passes during a forced Vercel token refresh, the call keeps the 409 `oauth_refreshed_retry_required` stop. `tool-gateway.ts` adds this reason to the deliberate stops, so the deadline does not change it to a 504 `tool_timeout`. ## Verification Head SHA: `25c6e703d1ded895989e9d2fcd6d6352a4f851f5`. It is the previous head `7836447f2` plus commit `25c6e703d`, which keeps the `oauth_refreshed_retry_required` reason when the call deadline passes during a token refresh. At this head, run from `server` with pnpm 9.15.4: `pnpm typecheck`: no errors; `mcp-http.test.ts`, `tool-gateway.test.ts` and `tool-gateway-service.test.ts`: 307 passed. Without the one-line change in `tool-gateway.ts`, the new test "keeps the explicit retry reason when the call deadline passes during the Vercel token refresh" fails: it receives 504 `tool_timeout`, not 409 `oauth_refreshed_retry_required`. Previous head: `7836447f24e2e6e9e0c402d4644d1f7f69b75cb8`. It is an ordinary merge of master `76994c50275dbc005cc248e11189a38baa21719f`, which includes #19877, into the previous head `00150e59c`. At this head, run from `server` with pnpm 9.15.4: `pnpm typecheck` and the test commands below. Results: typecheck: no errors; the 2 test files above: 116 passed. Previous head: `00150e59c1`. It merges master `18b6b71d8`. Run from `server`: ```sh pnpm exec vitest run src/__tests__/mcp-http.test.ts src/__tests__/tool-gateway.test.ts ``` - With this branch: 116 passed. With tool-gateway-service.test.ts and remote-mcp-socks-proxy.test.ts added: 315 passed in 4 files. - With `services/mcp-http.ts`, `services/remote-http-fetch.ts` and `services/tool-gateway.ts` from master and the test files from this branch, measured at `717e6c77c`: 7 failed, 106 passed. The failing tests are the six "records remote SSE ... without confusing a call deadline with a broken transport" scenarios and "retains consumed SSE progress when a partial event fails and releases the reader". - The existing master tests "keeps a remote MCP connection healthy and starts a fresh MCP session after a tool call timeout" and "... after a response body timeout" pass with this branch. - Typecheck: `pnpm --filter @paperclipai/server typecheck` reports 29 errors, all "Cannot find module @paperclipai/plugin-sdk" and the implicit any that follows, in the sandbox provider plugin packages of my local install. None is in a file that this pull request touches. - I ran the tests with an embedded PostgreSQL from the repository dependencies. The full upstream suite runs in CI. I ran only the test files named above. ## Risks - Medium risk for slow tools: a remote MCP call that uses the default deadline can now run for up to 30 seconds instead of 10 before it fails with `tool_timeout`. An explicit `timeoutMs` still applies. - The call evidence is larger: it can hold a redacted error chain and its stack traces. All strings go through `redactSensitiveText`. - No automatic replay: a timed-out `tools/call` is not sent again, because it can have changed data in the remote app. ## Model Used - Original change: the model that wrote it was not recorded. Port to current master, the adaptation to the connection health policy of master, and this description: Claude Opus 5.5 (claude-opus-5-5), with tool use and terminal code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no user-facing documentation change needed) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:19:08 -06:00
DottaandClaude Opus 5.5	ac0d0374f2	perf(issue-detail): measure task opens on real instances (#20276 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People open a task from their issues list to see what an agent is doing and to reply > - On a busy staging instance, the composer and the latest messages take 1.1–2.0 s to appear after a click. For one task with a slow run log, they took 13.9 s > - Local fixtures do not reproduce this. They do not have real network round trips, real server contention, or real object-store latency > - This pull request adds the measurement we need: a browser harness that times a task open on a real instance, and a Server-Timing entry on every API response that separates server time from network time > - It also records the staging baseline and the improvement plan > - The benefit is that each later performance change can be measured on real data with the same tool, before and after deploy ## Linked Issues or Issue Description Problem or motivation Opening a task from the issues list is slow on instances with real data. The composer and the latest chat messages can take seconds to appear. We could not measure this on staging in a repeatable way. We also could not tell server time from network and queueing time for most API requests. Proposed solution Add a reusable browser harness for task opens, and add a `paperclip_api` Server-Timing entry to every API response. Alternatives considered Local Playwright fixtures only. They do not reproduce real network round trips, server contention or object-store latency, so they did not show the delays that people see. Per-endpoint Server-Timing only. It does not cover the many other requests that run during a task open. Roadmap alignment This is measurement for existing task-loading behaviour. It does not add product features. The staging baseline and the measured causes are in `doc/plans/2026-11-09-task-loading-performance.md`. Follow-up pull requests change the loading behaviour itself, starting with #20277. ## What Changed - Added `server/src/middleware/api-server-timing.ts`. Every `/api` response now has a `paperclip_api;dur=<ms>` Server-Timing entry. The value is the server time from request arrival until headers are written, including admission waits. The entry is appended, so the existing `paperclip_issue` phases stay first. - Registered the middleware first in `server/src/app.ts`. - Added `tests/perf/issue-detail/staging-click-harness.js`. It clicks a task row and records, frame by frame, when the composer and the latest message become visible, layout shifts, movement of the latest message, and the requests that run before the page is ready. The summary labels each sample instead of using the task reference. It replaces query values and every path segment that contains a digit. A fetch that is still running when the sample ends is listed as pending. - Documented the harness in `tests/perf/issue-detail/README.md`. - Added the dated plan `doc/plans/2026-11-09-task-loading-performance.md` with the staging baseline, the causes, and the prioritized slices. ## Verification - `cd server && npx vitest run src/__tests__/api-server-timing.test.ts`. 3 tests pass. They cover the JSON responses, the time spent in earlier middleware, preserving route phases, and explicit `writeHead` calls. - `tests/perf/issue-detail/issue-detail.perf.spec.ts` reads the first `dur=` value. That value is still `paperclip_issue`. - The harness was used on staging to capture the baseline in the plan document. ## Risks - Low risk. The header has only a duration, with no request, user or company data. - The middleware wraps `res.writeHead` once for each request. This is the same pattern that `on-headers` uses. - Server-Timing is visible to same-origin scripts only. No `Timing-Allow-Origin` header is added. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, browser automation, file editing). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:06:37 -06:00
DottaandClaude Opus 5.5	fb88a89f04	fix(chat): tell chat agents how to link a case page while Cases is enabled (#20265 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat answers questions about company records and links each record it cites > - Chat guidance gives link formats for tasks, approvals, runs and agent profiles > - Cases (an experiment) have their own page at `/<prefix>/cases/<identifier>`, but the guidance does not name it > - So an agent that answers a question about a case's fact history names the case but cannot link it > - This pull request adds a Cases directive to chat guidance, only while the Cases experiment is on > - The benefit is that a person can open the case, where its facts and audit history are, from the reply ## Linked Issues or Issue Description What happened In a chat eval with Cases enabled, a person asked who changed a case's diagnosis. The agent read the case audit and the task's document revisions and answered correctly. It wrote "case COM-C1" as plain text. It linked only the task. The guidance in the chat prompt has no link format for a case, and the agent does not guess routes. Expected behavior When a reply concerns a case (its facts, fact-change history or handoffs), the agent links the case page as a Markdown link, for example `[RUN-C1](/<prefix>/cases/RUN-C1)`, and also links the tasks it cites. When the Cases experiment is off, the prompt does not mention cases. Steps to reproduce 1. Enable the Cases experiment. 2. Create a case with a primary task, and change one of its facts. 3. In Agent Chat, ask "Who changed the diagnosis, and what did it say before?" 4. Before this change, the reply names the case identifier but has no link to the case page. ## What Changed - `server/src/services/agent-conversations.ts`: new `CASES_CHAT_DIRECTIVE`. It says what a case is, names the `/<prefix>/cases/<case identifier>` page, and asks for a Markdown link when a reply concerns a case. - `server/src/services/heartbeat/task-markdown.ts`: new `casesEnabled` input. A conversation prompt includes the directive under "Cases:" only when it is true. - `server/src/services/heartbeat/task-context.ts`: sets `casesEnabled` from the `enableCases` experimental setting, the same way as `statusCardsEnabled`. - `server/src/services/heartbeat/task-markdown.test.ts`: new test. The directive appears for a standing chat with Cases on. It does not appear with Cases off or for a non-chat task. The test also checks that the board route `cases/:caseIdentifier` exists in `ui/src/App.tsx`. ## Verification - `cd server && npx vitest run src/services/heartbeat/task-markdown.test.ts` passes (18 tests). - `npx tsc --noEmit -p server` passes. - The `cases.case-history` chat eval uses this guidance. Its rerun is on branch `chat-eval/case-unclear-9`. ## Risks - Low risk. The change adds prompt text only when the Cases experiment is on. It does not change any API or permission. - The prompt for chats on instances with Cases enabled is slightly longer. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), 1M context, extended thinking, with tool use (shell, file edit, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:03:16 -06:00
DottaandClaude Opus 5.5	1004e55180	fix(recovery): stop stall wakes from failing setup on a held workspace (#20273 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Recovery tells a waiting parent or dependent when a child or blocker stalls on the board (`issue_blocker_stalled`) > - A legacy child that an older release bound to its parent's remote workspace failed its save, and its save recovery was exhausted. Its ended run still holds the shared workspace, days past the one-hour holder cutoff > - Workspace reservation refuses a run on such a hold and fails setup with "The workspace still has an unfinished writer or save operation." Every stall wake of the parent created a run that could only fail this way > - The stall sweep treated each failed run as an undelivered wake and retried it every ten minutes, with no backoff and no limit. On staging this made 72 failed runs (58 of 60 failures since a deploy), in pairs every ten minutes > - This pull request makes the stall sweep hold a wake while the waiter's workspace hold refuses a run, and bounds retries of a wake whose run fails > - The benefit is that a held workspace no longer produces a failed run every ten minutes, and the wake still arrives once the hold settles ## Linked Issues or Issue Description What happened? A parent task was woken by `issue_blocker_stalled` twice every ten minutes. Each run failed in setup (`setup_failed`, `executionRecovery.kind=bootstrap`, `providerWorkStarted=false`) with "The workspace still has an unfinished writer or save operation." The parent's remote workspace was held by a child's ended writer run whose save recovery was exhausted (`native_workspace_sync_out_retry_exhausted`, last failure `agent_stop_unconfirmed`). The child shared the parent's workspace because an older release created it before subtasks got their own workspace (#19860). Expected behavior A wake that cannot start because its workspace is held must not create a failed run every time. It should wait until the hold settles. Automatic retries must be bounded. Root cause - `server/src/services/heartbeat/workspace-preparation.ts` rethrows `RemoteWorkspaceWriterActiveError` as a setup failure when the ended writer is silent past `WORKSPACE_BUSY_HOLDER_STALE_AFTER_MS` (documented, intended). - `server/src/services/recovery/blocker-stall-wakeups.ts` retried every non-delivered wake row (including `failed`) after `REFUSED_WAKE_RETRY_MS` (10 minutes) with no backoff and no limit, and it did not check whether the waiter could start at all. ## What Changed - `remote-workspace-writer-hold.ts`: add `remoteWorkspaceHoldRefusesStart`, the one rule for a hold that reservation refuses outright (unattributed writer, or a writer silent past the cutoff with no live owner fence). - `workspace-preparation.ts`: use that rule instead of an inline copy, so reservation and wake senders cannot drift. - `blocker-stall-wakeups.ts`: before waking a waiter, check for proof that its run would be refused. The proof is that the assignee's own latest run on the task failed setup with the unfinished-writer error, and the bound workspace hold still refuses a run. Then do not wake it; count it as deferred and re-check at most hourly. A different assignee (for example a controller-only agent that never uses the workspace), a later run, or a settled hold lets the wake through. Reservation already tried and failed the unchanged-writer release for that run, so the sweep does not repeat that proof. - `blocker-stall-wakeups.ts`: count failed runs that carry the stall (the keyed wake's run and keyless resource-wait retries that keep its context). Retry after 10, then 20 minutes, and stop after the third failed run. Refusals (wake-on-demand off, agent paused) keep the 10-minute retry. A new stall (new recovery action) wakes again. - `remote-workspace-writer-hold.ts` / `remote-workspaces.ts`: share the unfinished-writer error text as one constant. - `doc/execution-semantics.md`: document both rules. ## Verification - Red before the fix (`server/src/__tests__/issue-blocker-stall-wakeups.test.ts`): 3 new tests failed. The held-workspace test got a delivered wake; the backoff test got a third wake 11 minutes after the second failure. - Green after the fix: `npx vitest run src/__tests__/issue-blocker-stall-wakeups.test.ts` passes 22/22. Added after review: a waiter with no refused run, or a reassigned waiter, is still woken; failed resource-wait retries count toward the limit. - The new tests use the staging shape: a legacy `shared_workspace` remote workspace owned by the parent, a child bound to it, the child's failed native writer run ended four days ago with `native_workspace_sync_out_retry_exhausted`, a dirty authority, and the child's board-owned exhausted recovery action. The parent's own earlier run failed setup on the hold, as on staging. They also prove that a writer inside the cutoff still gets its wake (the run parks in a workspace wait), and that the wake is delivered once the hold settles. - Related suites pass: `heartbeat-remote-workspace-busy.test.ts`, `heartbeat-dependency-scheduling.test.ts`, `native-cancelled-workspace-preservation.test.ts`, `src/modules/run-dispatch`. - `npx tsc --noEmit -p server` passes. - `pnpm verify` exits 2 because `doc/execution-semantics.md` is a global CI input; required CI supplies the broad coverage. ## Risks - Low. No schema or persisted-shape change. Saved state from the previous release (existing wake rows, existing workspace authority) is read as-is: earlier `failed` stall wake rows count toward the new three-run limit, which stops the current loop at once after deploy. - A parent whose workspace stays held is not woken for the stall until the hold settles. The stall is still listed on its page and the child's recovery action stays with the board. - Not fixed here: the exhausted save hold itself is never settled automatically (`agent_stop_unconfirmed` exhaustion in workspace recovery), and legacy children that still share a parent's workspace are not moved. Both touch workspace recovery code that is under concurrent change and are left for a follow-up. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser read-only inspection of staging). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 08:28:21 -06:00
DottaandClaude Opus 5.5	2dd7a98753	fix(recovery): return a settled restore hold's warm sandbox to reuse (#20278 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Recovery tells a waiting parent or dependent when a child or blocker stalls on the board (`issue_blocker_stalled`) > - A legacy child that an older release bound to its parent's remote workspace failed its save, and its save recovery was exhausted. Its ended run still holds the shared workspace, days past the one-hour holder cutoff > - Workspace reservation refuses a run on such a hold and fails setup with "The workspace still has an unfinished writer or save operation." Every stall wake of the parent created a run that could only fail this way > - The stall sweep treated each failed run as an undelivered wake and retried it every ten minutes, with no backoff and no limit. On staging this made 72 failed runs (58 of 60 failures since a deploy), in pairs every ten minutes > - This pull request makes the stall sweep hold a wake while the waiter's workspace hold refuses a run, and bounds retries of a wake whose run fails > - The benefit is that a held workspace no longer produces a failed run every ten minutes, and the wake still arrives once the hold settles ## Linked Issues or Issue Description What happened? A parent task was woken by `issue_blocker_stalled` twice every ten minutes. Each run failed in setup (`setup_failed`, `executionRecovery.kind=bootstrap`, `providerWorkStarted=false`) with "The workspace still has an unfinished writer or save operation." The parent's remote workspace was held by a child's ended writer run whose save recovery was exhausted (`native_workspace_sync_out_retry_exhausted`, last failure `agent_stop_unconfirmed`). The child shared the parent's workspace because an older release created it before subtasks got their own workspace (#19860). Expected behavior A wake that cannot start because its workspace is held must not create a failed run every time. It should wait until the hold settles. Automatic retries must be bounded. Root cause - `server/src/services/heartbeat/workspace-preparation.ts` rethrows `RemoteWorkspaceWriterActiveError` as a setup failure when the ended writer is silent past `WORKSPACE_BUSY_HOLDER_STALE_AFTER_MS` (documented, intended). - `server/src/services/recovery/blocker-stall-wakeups.ts` retried every non-delivered wake row (including `failed`) after `REFUSED_WAKE_RETRY_MS` (10 minutes) with no backoff and no limit, and it did not check whether the waiter could start at all. ## What Changed - `remote-workspace-writer-hold.ts`: add `remoteWorkspaceHoldRefusesStart`, the one rule for a hold that reservation refuses outright (unattributed writer, or a writer silent past the cutoff with no live owner fence). - `workspace-preparation.ts`: use that rule instead of an inline copy, so reservation and wake senders cannot drift. - `blocker-stall-wakeups.ts`: before waking a waiter, read its bound remote workspace hold. When the hold refuses a run, do not wake it; count it as deferred and re-check at most hourly. A workspace with an unchanged-release baseline is left to reservation, which can release it. - `blocker-stall-wakeups.ts`: a stall wake whose run failed is retried after 10, then 20 minutes, and not after the third failed run. Refusals (wake-on-demand off, agent paused) keep the 10-minute retry. A new stall (new recovery action) wakes again. - `doc/execution-semantics.md`: document both rules. ## Verification - Red before the fix (`server/src/__tests__/issue-blocker-stall-wakeups.test.ts`): 3 new tests failed. The held-workspace test got a delivered wake; the backoff test got a third wake 11 minutes after the second failure. - Green after the fix: `npx vitest run src/__tests__/issue-blocker-stall-wakeups.test.ts` passes 20/20. - The new tests use the staging shape: a legacy `shared_workspace` remote workspace owned by the parent, a child bound to it, the child's failed native writer run ended four days ago with `native_workspace_sync_out_retry_exhausted`, a dirty authority, and the child's board-owned exhausted recovery action. They also prove that a writer inside the cutoff still gets its wake (the run parks in a workspace wait), and that the wake is delivered once the hold settles. - Related suites pass: `heartbeat-remote-workspace-busy.test.ts`, `heartbeat-dependency-scheduling.test.ts`, `native-cancelled-workspace-preservation.test.ts`, `src/modules/run-dispatch`. - `npx tsc --noEmit -p server` passes. - `pnpm verify` exits 2 because `doc/execution-semantics.md` is a global CI input; required CI supplies the broad coverage. ## Risks - Low. No schema or persisted-shape change. Saved state from the previous release (existing wake rows, existing workspace authority) is read as-is: earlier `failed` stall wake rows count toward the new three-run limit, which stops the current loop at once after deploy. - A parent whose workspace stays held is not woken for the stall until the hold settles. The stall is still listed on its page and the child's recovery action stays with the board. - Not fixed here: the exhausted save hold itself is never settled automatically (`agent_stop_unconfirmed` exhaustion in workspace recovery), and legacy children that still share a parent's workspace are not moved. Both touch workspace recovery code that is under concurrent change and are left for a follow-up. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, browser read-only inspection of staging). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 08:28:04 -06:00
DottaandClaude Opus 5.5	1b09920a64	fix(issues): refuse agent status changes that detach another run's live execution (#20271 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A task that an agent run executes holds a checkout lock and an execution lock (`checkoutRunId`, `executionRunId`) > - A status change away from `in_progress` clears these locks. It does not stop the run that holds them > - Agents cannot stop another agent's run: run cancel and the task-tree pause are board-only > - But an agent that can edit a task (for example a lead with the manage-active-checkouts grant over its report) can change its status. The run then keeps running and is no longer bound to its task > - In chat evals, a lead "paused" a launch tree. It moved running subtasks to backlog. The runs kept running, detached > - This pull request refuses an agent's status change when a different queued or running run holds the task's lock. The denial is final, links the task, and names the person's controls > - The benefit is that agents cannot interrupt another agent's run through a status edit, and runs are not left orphaned ## Linked Issues or Issue Description Refs #20261 (chat guidance sends a task-tree pause to the person's Pause subtree control). Refs #19394 (resume restarts the work a subtree pause interrupted). What happened An agent changed the status of a task while another agent's run executed it. The request succeeded. The server cleared `checkoutRunId` and `executionRunId`. The run continued, but it was no longer bound to the task. In the chat evals `steering.stop-versus-pause` and `task-coordination.pause-tree`, a lead moved running subtasks to backlog to "pause" a launch. Their runs kept running, detached. Expected behavior An agent cannot get the effect of interrupting another agent's run. When a different live run holds the task's lock, the agent's status change is refused before any write. The refusal says it is final. It sends the person to the task and names the controls that really stop or hold the work: the run's Stop or Interrupt, or More task actions, then Pause subtree on the parent task. The lock-holding run, board users, and paths that stop the run themselves continue to work. Steps to reproduce 1. Create agent A and agent B, with B reporting to A (A gets `tasks:manage_active_checkouts` through the manager chain). 2. Give B an `in_progress` task. Set `checkoutRunId` and `executionRunId` to a `running` heartbeat run of B. 3. As A (agent JWT with its own run), send `PATCH /api/issues/:id` with `{ "status": "backlog" }`. 4. Before this change: `200`, the status is `backlog`, both lock fields are `null`, and B's run is still `running`. ## What Changed - `server/src/routes/issues.ts`: new `assertAgentStatusChangeKeepsLiveRunBound`. For an agent actor, it refuses a status change when `executionRunId` or `checkoutRunId` names a different run in the same company that is `queued` or `running`. It runs in `PATCH /issues/:id` after the final status and assignee are known (execution-policy transitions included) and before any run stop or write. It also runs in recovery resolution when the source status changes. - The guard applies to any status other than `in_progress`, also to the current status, because the service clears the locks for those too. - The write is conditional on the lock the guard checked. The route re-reads the lock and passes it to `svc.update` as `expectedExecutionLock`. The service compares it under the row lock and returns `409 issue_execution_lock_changed` if a checkout landed in between. - These cases stay allowed: the lock-holding run itself, board users, requests with no status, a change to `in_progress`, `cancelled` (the route cancels the run), reassignment (the route stops the run first), `done` through PATCH while a Runner goal is active (terminalization pauses the goal and stops the run; recovery resolution does not get this exception), and a lock holder that already finished. - `packages/shared/src/issue-write-denial.ts`: new denial code `issue_status_change_live_run` (409). The copy says the refusal is final, links the task as `[identifier](/prefix/issues/identifier)`, and names Stop or Interrupt and More task actions, then Pause subtree. A new optional `issuePath` context field carries the link. The UI notice renders it through the existing generic path. - `doc/SPEC-implementation.md`: documents the rule in the checkout contract section. - Paths checked with no change needed: there is no bulk issue update route; tree holds and run cancel are board-only; `POST /issues/:id/release` already lets only the assignee's checkout run release; API-catalog actions call the same PATCH route; MCP status changes already refuse with `MCP_ACTIVE_EXECUTION`. ## Verification - Red on master, green with this change: - `server/src/__tests__/issue-stale-execution-lock-routes.test.ts` (embedded Postgres, real issue service): a manager moves its report's running task to backlog. On master: `200`, status `backlog`, `checkoutRunId`/`executionRunId` changed to `null`. With the fix: `409 issue_status_change_live_run`, row unchanged, run still `running`. - `server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts`: refusals for `backlog`, `todo`, `blocked`, `in_review`, `done`, a queued lock holder, and a repeated current status (red on master). Allowed cases: field edits, the lock-holding run, board, `cancelled` (run is cancelled), and finished lock holders; an allowed change carries `expectedExecutionLock`. - Same embedded-Postgres file: `svc.update` with an `expectedExecutionLock` that differs from the row returns `409 issue_execution_lock_changed` and leaves the row unchanged. - Also passed locally: `issue-closed-workspace-routes`, `issue-comment-reopen-routes`, `issue-recovery-actions`, `issue-scheduled-retry-routes`, `low-trust-red-team-routes`, `issue-update-comment-wakeup-routes`, `external-object-routes`, `packages/shared` `issue-write-denial`, `ui` `IssueWriteDenialNotice`. - `tsc --noEmit` shows no errors in the changed files. Full typecheck, build and test are left to CI. ## Risks - Behavior change for agents: a lead or CEO that used status edits to take over a report's running task now gets a 409. The supported paths (comment, cancel, reassign, or the person's Stop or Pause subtree) still work. - A queued run that holds the lock also blocks the change. This is deliberate: the queued run would otherwise start later with no task bound to it. - Agents can still cancel another agent's run by cancelling its task. That is existing, documented behavior ("task cancellation cancels the intent"), and this PR does not change it. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 08:21:40 -06:00
DottaandClaude Opus 5.5	6459488984	fix(native): open a remote replacement as a new runner after archiving the reattached one (#20242 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - After a restart, a remote (Daytona) run can resume by reattaching to its surviving sandbox runner. If provider recovery fails and the checkpoint allows it, the run opens a replacement provider session instead. > - Before a replacement, the executor archives the local controller state and the remote session root as a continuity break, then opens a new session. > - On a reattach backend, the replacement still adopted the archived runner and asked for its old authority, so it failed. The archive had already moved the remote state, so the next retry could not verify the runner and the issue waited 17 minutes on an unconfirmed stop. > - Review showed that a reattach backend fixes many launch inputs once at creation (runner binary, provider pack and companions, artifacts, live-recovery mode, authority, spawn barrier), so resetting them after the archive does not scale. > - This pull request builds the replacement on a new backend for the same run, created under a `bootstrap_incomplete` claim after the archive, so every launch input resolves through the ordinary fresh path. > - The benefit is that a failed reattach after a restart falls back to a working replacement instead of a stuck issue. ## Linked Issues or Issue Description What happened? In restart-chaos Daytona K13 storm wave seed 9503, task t36 graded R1 (no rest within budget). A K2 drain suspended run a53dd948's sandbox runner (pid 589) during startup. On restart the run claimed `reattach_remote_runner`. The adopted runner authenticated and restored, then its connection closed, and provider recovery failed. The runtime opened a replacement: `provider.session.archive_before_replacement` archived the local and remote state, then the replacement failed with "PRP provider resume state is unavailable". The same-run retry at 10:25:50 then failed with `runner_remote_state_unavailable: exit=1` (the remote `runner-state.json` had moved), and file preservation deferred on `agent_stop_unconfirmed` until the wave ended. Expected behavior After the continuity archive, the replacement opens a new runner for this run, as it does for every other claim kind. Steps to reproduce 1. Run a remote (Daytona) native run and restart the server so the run claims `reattach_remote_runner`. 2. Make provider recovery fail on the adopted runner with a checkpoint that allows replacement. 3. See the replacement open on the adopted runner's authority and fail after the archive. Paperclip version or commit master at `465da4227a` ## What Changed - `native-session-executor.ts` `openReplacementSession`, on a `reattach_remote_runner` backend: archives the continuity state (local and remote, as before), then calls `createRunnerdBackendWithinSessionClaim` for the same run with the claim re-kinded to `bootstrap_incomplete` (minus `remote` and `stagedAttachment`) and without warm recovery, and opens the replacement on that backend. Root migration then quarantines the empty archived root (the #20241 rule) and every launch input resolves fresh. - A backend that adopts only through warm recovery (no claim to re-kind) throws `native_runner_replacement_requires_fresh_launch` before archiving, so its saved state stays intact. - Tests: a classic Daytona reattach backend for `codex` and `acpx` (Claude). The replacement comes from a second backend: the original backend's `openSession` is never called, and the new transport has no adopted runner, uses `turn-<runId>`/`item-<runId>`, has a new still-pending spawn barrier, and (for `acpx`) carries the packaged provider paths. The bundled-image replacement test now expects a new backend and its own pack admission, not the recovered session's cold admission. ## Verification - Red before the fix: both new replacement tests fail on master. They pass with the fix. - `native-session-executor.test.ts`, `native-recovered-owner-followup.test.ts`, `native-harness-filesystem-independence-continuation.integration.test.ts`: 1,398 passed. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server`: passes. ## Risks - Medium. The replacement now constructs a second backend inside the same session claim. The new backend's tool-authority epoch supersedes the reattach backend's, which is being abandoned anyway. - A warm-recovery-only replacement now fails with a named error instead of archiving and then failing. Its saved state stays intact for the next recovery. - Previous-release story: no saved state or schema changes. - Not covered: why provider recovery on the suspended, adopted runner failed in t36 in the first place. This PR makes the fallback work. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 08:12:24 -06:00
DottaandPaperclip	a3a25ce6c9	fix(workspaces): stop running Paperclip's worktree setup before every agent turn (#20269 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before an agent turn starts, the harness prepares the task's git worktree > - The harness runs a repository's `scripts/provision-worktree.sh` when the project sets no `provisionCommand` and the file exists > - The harness runs it on every turn, both when it creates the worktree and when it reuses it > - In the Paperclip repository, that script writes the worktree's isolated instance config and runs `pnpm install` > - Its "install needed" check compares against every `node_modules` directory in the base checkout, including other worktrees, so it reinstalls on every turn > - This pull request renames the Paperclip script so that the harness default does not find it in this repository > - The benefit is that Paperclip agent turns no longer wait on a 6–9 second install. Other repositories keep the harness feature. ## Linked Issues or Issue Description What existing behavior does this improve? Time to first token for agent turns in managed worktrees of the Paperclip repository. Subsystem affected Execution workspace provisioning (`server/src/services/workspace-runtime.ts`) and the repository script `scripts/provision-worktree.sh`. Current behavior Every agent turn in a Paperclip worktree first runs `bash ./scripts/provision-worktree.sh`. In a local test, a `pnpm install --frozen-lockfile` with nothing to change took 6.1 s and 6.3 s, and a full install from a warm store took 9.0 s. The script's fingerprint skip does not prevent this, because the script also requires every base-checkout `node_modules` path (up to depth 4) to exist in the worktree. On a developer machine that list had 1,735 entries, including other worktrees under `.claude/worktrees/` and `.worktrees/` and pnpm's internal `node_modules/.pnpm//node_modules`. A managed worktree never contains the other worktrees' paths, so the script always decides to reinstall. Proposed behavior* The Paperclip repository does not ship a file at `scripts/provision-worktree.sh`. The setup script becomes `scripts/setup-paperclip-worktree.sh`. People can run it by hand or set it as the project `provisionCommand` to opt in. The harness default stays for other repositories. Reason and benefit Agents install dependencies when they need them (`node scripts/agent-preflight.mjs`, as `AGENTS.md` says). Many turns never run code. Instance setup stays available on demand: `scripts/provision-worktree-runtime.sh` runs the setup script when a dev server starts in a worktree that has no config. Database seeding was already deferred. Breaking changes A project that explicitly sets `provisionCommand` to `bash ./scripts/provision-worktree.sh` for this repository will fail with "file not found". Change it to `bash ./scripts/setup-paperclip-worktree.sh` or clear it. ## What Changed - Rename `scripts/provision-worktree.sh` to `scripts/setup-paperclip-worktree.sh`. The content is unchanged. - `scripts/provision-worktree-runtime.sh` calls the new name when the worktree has no config. - Tests read the real script from its new path (`provision-worktree-self-heal`, `workspace-runtime`, `worktree-seed-server-spawn`). Server fixtures still copy it into their test repositories as `scripts/provision-worktree.sh`, so the harness default stays covered. - `doc/DEVELOPING.md` says that the Paperclip repository deliberately has no `provision-worktree.sh`, and tells people how to opt in. ## Verification - `node --test scripts/__tests__/provision-worktree-self-heal.test.mjs`: 33 passed, 1 skipped (the skip already existed). - `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts src/__tests__/heartbeat-workspace-source-trust.test.ts src/__tests__/execution-workspace-reopen.test.ts`: 242 passed on the rebased head. - `cd server && npx vitest run src/__tests__/worktree-seed-server-spawn.test.ts`: 2 passed. - Manual: in a Paperclip worktree, start an agent turn. The run log does not show a `workspace_provision` operation for the script. ## Risks - Explicit `provisionCommand` settings that name the old path will fail. See "Breaking changes". - Open pull requests that edit `scripts/provision-worktree.sh` (for example #16213, #11335, #11733, #19764) will conflict on the rename. Git rename detection should carry most edits over. - An agent that runs tests in a new worktree without first running preflight will see a missing-module error. `AGENTS.md` already tells agents to run `node scripts/agent-preflight.mjs`. - The script's install check still has the bug described above. It now runs only on demand. A separate fix can address it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, file edits). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-09 14:03:24 +00:00
DottaandClaude Opus 5.5	33a23e0246	fix(chat): send a task-tree pause to the person's Pause subtree control (#20261 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A person can pause a whole task tree from the parent task page: More task actions, then Pause subtree > - The pause holds every task under the parent, stops their active runs and keeps assignees; Resume subtree restarts the interrupted work (#19394) > - The tree-control routes are board-only, so a chat agent cannot apply this pause > - Asked to pause a launch tree, chat agents imitated the pause: they moved running subtasks to backlog, cleared their execution and told the person the work was paused > - This pull request tells chat agents, in guidance they always load, to send the person to Pause subtree and not to imitate it > - The benefit is that the person gets a real hold that keeps assignments and resumes cleanly, and the reply does not claim a pause that does not exist ## Linked Issues or Issue Description Refs #20252 (marks the tree-control routes board-only in the API catalog and names the page). Refs #19394 (resume restarts interrupted work). What happened A person asked a chat lead "Pause the whole Launch task tree" or "Pause all work on the launch until tomorrow". In three of four evaluated runs the lead did not use or point to the tree pause. It changed the two running subtasks from in_progress to backlog, which cleared their execution, and replied "All launch work is paused". Its two native runs kept running, and the run cancel calls were denied. In one run it also said "I scheduled a check for 2026-11-10 12:10 UTC to resume the launch". No tree hold existed, so Resume subtree later had no interrupted work to restart. The agent's reasoning shows it decided early that "pause" meant backlog ("No issue-level pause endpoint. So pause = set to backlog"). It did not reach the board-only catalog entry for the tree hold, even with #20252 applied. Expected behavior The agent finds the parent task and replies with a Markdown link to it. It tells the person to use More task actions, then Pause subtree on that page, and Resume subtree later. It does not change task statuses or assignees, cancel runs, pause agents, or create tasks or routines to imitate the pause. It does not claim a scheduled resume, because Paperclip has none. Steps to reproduce 1. Create a parent task with two child tasks whose assigned agents are running. 2. In a chat with the company lead, send "Pause the whole Launch task tree." 3. Read the reply and the task activity: before this change the lead often moves the running children to backlog and says the tree is paused. ## What Changed - `server/src/services/agent-conversations.ts`: add one paragraph to the chat directive. It says that pausing, stopping or resuming a task tree is a person's control on the parent task page (More task actions, then Pause subtree, and later Resume subtree). It says what the pause does, that there is no scheduled resume, and that the agent must not imitate the pause with status, assignee, run, agent or routine changes. The agent links the parent task and replies. - `server/src/__tests__/agent-conversations.test.ts`: add a regression test for the directive text and for its presence in the built chat prompt. ## Verification - `cd server && npx vitest run src/__tests__/agent-conversations.test.ts src/__tests__/board-access-denial.test.ts src/services/heartbeat/task-markdown.test.ts src/__tests__/heartbeat-chat-task-link.test.ts src/__tests__/heartbeat-context-summary.test.ts src/services/native-runtime/current-wake-comments.test.ts src/__tests__/codex-local-execute.test.ts src/__tests__/claude-local-execute.test.ts` passes. - End to end: the chat evals `task-coordination.pause-tree` and `steering.stop-versus-pause` run this change on an eval branch. A human then clicks Pause subtree, and the eval checks the hold, the two acknowledged native stops, and that Resume subtree restarts only the interrupted branches. ## Risks - Low risk. The change is prompt text only. It adds about 900 characters to every chat prompt. - An agent that has a real need to change one child task's status in the same conversation can still do it when the person asks for that change. The paragraph covers requests to pause or stop a tree. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:33:40 -06:00
DottaandClaude Opus 5.5	ad440d4e91	fix(chat): name the managed-profile CLI and the dev-only labs in agent guidance (#20236 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat answers a person's requests with a chat agent that reads the API catalog and the chat guidance > - Two facts are missing from what that agent reads > - Claude Managed Agent profiles have no web page. A board user creates them with the `paperclipai managed-agent setup` CLI command. The catalog does not say this, so agents tell people to use "the Paperclip web app" > - Developer labs, such as `/<prefix>/dev/task-chat-lab`, exist only in a development build. Nothing says this, so agents probe the production bundle in a headless browser for many steps and then ask the person for a "dev-mode URL" > - This pull request adds the two facts where the agent already looks > - The benefit is a correct next step for the person, in fewer agent steps ## Linked Issues or Issue Description What happened An agent was asked to "Set up an Anthropic managed agent for this company". It correctly said the action is board-only. Then it told the board user to create the profile "in the Paperclip web app". That page does not exist. An agent was asked to "Open the task-chat interaction lab from this link". The link went to a production instance. The agent used 56 steps, mostly bash and headless Chrome, and did not tell the person that the lab runs only under `pnpm dev` in their own checkout. Expected behavior For managed agents, the agent names the CLI command that a board user runs. It also says that no web page exists. For developer labs, the agent says the lab is development-only. It tells the person to run `pnpm dev` in their own checkout and open the route there. It does not probe the instance. Steps to reproduce 1. In Agent Chat on a production build, ask "Set up an Anthropic managed agent for this company". 2. Read the reply. It points to a web page that does not exist. 3. Attach the link `http://<host>/<prefix>/dev/task-chat-lab` and ask "Open the task-chat interaction lab from this link". 4. Look at the run. The agent probes the bundle and does not give the `pnpm dev` path. ## What Changed - `server/src/routes/openapi.ts`: the GET and POST managed-agent-profile routes now say that no web page exists. The POST description now names `paperclipai managed-agent setup --company-id <companyId> --profile-key <key> --display-name <name> --api-key-secret-id <secretId> --acknowledge-retention`, the `ANTHROPIC_API_KEY` requirement, the `--probe` option and the Secrets page. - `server/src/services/agent-conversations.ts`: the chat directive has a new short paragraph. It says some developer labs, including the task-chat lab, are registered only in a development build, it tells agents not to probe, and it gives the `pnpm dev` path with the route. It also says the design guide and ux-lab pages exist in every build, so agents open those normally. - Tests for both texts in `openapi-routes.test.ts` and `agent-conversations.test.ts`. ## Verification - `cd server && pnpm exec vitest run src/__tests__/openapi-routes.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/board-access-denial.test.ts src/__tests__/codex-local-execute.test.ts`: 142 tests pass. - After merging master (keeping master's ready-made teams paragraph) and the review fixes: `cd server && pnpm exec vitest run src/__tests__/agent-conversations.test.ts src/__tests__/openapi-routes.test.ts src/services/native-runtime/runner-api.test.ts`: 2164 tests pass. - The change is also cherry-picked onto the eval branch `chat-eval/case-keyword-graders`. The live Runner E2E cells for `agents.managed-profile` and `developer-labs.gated-lab` run there. ## Risks - Low risk. The change is text in the API catalog and in the chat guidance. It does not change behavior or permissions. - The chat directive is slightly longer, so each chat prompt uses a few more tokens. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:15:16 -06:00
DottaandClaude Opus 5.5	2b461b2f58	fix(native): require publication for files named under a Deliverable label (#20231 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native agents finish a task with a completion report, and a server gate checks that requested files are published to the task > - The gate finds a file request only from a creation verb, such as "Write report.pdf" > - Delegated briefs often list the output under a "Deliverable:" label instead, and the gate did not know `.html` or `.css` names > - So a worker can finish a website task with `index.html` only in its own workspace, and the person cannot open it > - This pull request makes a file named under a Deliverable label a requested file output > - The benefit is that the worker gets the existing "use register_deliverable" feedback before the task closes ## Linked Issues or Issue Description No public issue exists. Bug description: What happened A person asked the CEO: "Make me a simple website introducing our business." The CEO delegated a task. Its brief said "Deliverable: - A self-contained static website in this task's workspace: index.html plus a stylesheet (for example styles.css)". The worker wrote `index.html` and `styles.css` in its workspace. It did not call `register_deliverable`. It cited `index.html` and `styles.css` as completion evidence, and the gate accepted the report. The task was marked done with no attachment. The CEO then told the person that the files were "in the task workspace". Expected behavior A brief that names a file as its deliverable is a file request. The completion gate must reject a workspace-only file and tell the worker to publish it with `register_deliverable`. Steps to reproduce 1. Call `explicitlyRequestsFileOutput` with the brief above. 2. See that it returns `false`, so `validateNativeDeliverableEvidence` accepts the bare `index.html` evidence ref. ## What Changed - `explicitlyRequestsFileOutput` now also returns true when a `Deliverable:` / `Deliverables` label line (plain, bold or heading), or a bullet or numbered item in the list below it (blank lines between items allowed), names a file. Relative paths such as `dist/index.html` count. - Under the label, web page files (`.html`, `.htm`, `.css`) count with the existing file types. - Items that name an input ("a summary of input.csv", "a comparison of input.csv and baseline.csv"), describe a repository change, pull request, commit or branch, forbid the named file (`do not create report.pdf`), name a file to edit in place (`update styles.css`) without asking to publish it, send the file inline (`index.html and styles.css inline in the reply`), or forbid creating files, do not count. An inline or no-upload instruction about another output in the same item (`index.html and a short summary in the reply`, `index.html; do not upload it externally`) does not cancel the named file. The repository-change check ignores file names, so `branch-report.pdf` still counts. - The sentence-based detector is unchanged. - `doc/AGENT-ARTIFACTS.md` describes the label rule. ## Verification - `cd server && npx vitest run src/services/native-runtime/native-deliverable-feedback.test.ts src/services/native-runtime/native-runner-file-handoff.test.ts` (197 pass), then after the review fixes `native-deliverable-feedback.test.ts`, `native-completion-format-feedback.test.ts` and `chat-attachment-reuse.test.ts` (211 pass). The 13 first-round review cases fail on the original head. - New cases: the real delegated brief, inline labels, heading lists, numbered lists and bold labels require publication. Unlabelled text, inline-chat deliverables, input references, repository changes, text after the list ends, and "Deliverable quality ..." prose do not. - The chat eval `onboarding.first-work` exercises this path end to end on its candidate branch. ## Risks - Low risk. The change only adds a true result for labelled file outputs; an inline-only or no-file label item stays false, as the sentence detector already decided. - A labelled brief that names a file which is not a repository change now needs `register_deliverable` or an accessible work product. That is the documented delivery rule. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:15:11 -06:00
DottaandClaude Opus 5.5	68253e3175	fix(recovery): continue a run refused by an inert cleanup quarantine with no click (#20260 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a native session's cleanup fails unrecoverably, the runtime quarantines it in process. The next run for that task is refused with `native_session_cleanup_quarantined`. > - A board Retry or a new message already recovers from this. `verifyStoppedNativeSessionForContinuation` proves that the old runner and providers stopped, releases the quarantine and starts a fresh session. > - Without a click nothing recovers. The automatic safe-replacement sweep proves stopped sessions only with the Codex replacement verifier, which needs the failed run's own active turn. A run refused before launch never has one, so the sweep skips it on every pass. > - Restarts must recover on their own. This pull request lets the sweep use the same continuation proof for a cleanup-quarantined run. > - The benefit is that an inert quarantine turns into a fresh-session continuation with no click, and an unproven one still keeps its board Retry. ## Linked Issues or Issue Description What happened? In restart-chaos wave seed 9802 (local, G1, Claude), task t02's run 71f03e16 succeeded, but its runner cleanup could not prove it drained and was quarantined. Follow-up run 0b4b404f was refused before launch with `native_session_cleanup_quarantined`, and the task showed "Automatic recovery stopped" until someone pressed Retry. Expected behavior When the quarantined session's owner run and every provider process are proven stopped, recovery continues the task by itself in a fresh session, without charging an attempt. When that cannot be proven, the board Retry stays available. Steps to reproduce 1. Have a native run succeed while its runner cleanup is quarantined. 2. Let the next run for the task be refused with `native_session_cleanup_quarantined` before it launches. 3. Stop the old runner and provider processes, then run the safe-replacement sweep. No successor is created. Paperclip version or commit master at `8eb11fa8aa` ## What Changed - `native-session-executor.ts`: new `verifyStoppedNativeSessionForAutomaticReplacement`. It returns the existing replacement proof, or, for a `native_session_cleanup_quarantined` run with no replacement proof, the continuation proof that the board Retry already uses (Codex and ACPX, including an inherited owner run). - `server/src/index.ts`: the safe-replacement sweep uses that verifier. - The existing sweep then retires the quarantine, records `native.stopped_text_turn_verified` and queues a successor with `forceFreshSession: true` at the failed run's attempt number, so no attempt is charged. - `quarantined-task-retry.test.ts`: "recovers a quarantined $provider source that never launched with no click…" for Codex and ACPX (Claude). A stopped owner makes the sweep queue one fresh-session successor at the same attempt. A running owner (Codex) or a live ACPX agent process queues nothing and leaves the board Retry in place. ## Verification - Red before the fix: with the replacement-only proof, the stopped case queues no successor. It passes with the fallback, and the running-owner case passes on both. - `quarantined-task-retry` (35), `native-safe-replacement`, `native-replacement-evidence`: pass. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server`: passes. ## Risks - Low to medium. The fallback applies only to `native_session_cleanup_quarantined` runs. It uses the stricter continuation proof that already gates the board Retry: owner run settled, local leases released, runner and every recorded provider pid absent, and an unchanged retained state. The sweep keeps every other gate (task ownership, failure-block receipt, attempt limit, effect inventory). - Previous-release story: no schema or saved-state change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:12:07 -06:00
DottaandClaude Opus 5.5	1a007c4ef1	fix(accounting): settle a native run that only inherited its predecessor's runner identity as no provider work (#20250 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each finished run settles its accounting. A native run whose provider never started settles at once with no charge. Other runs wait for a usage receipt, bounded by a closure timeout. > - A company budget policy can block unpriced usage. Unpriced cost rows then trip a hard stop that pauses the company. > - A follow-up run that is refused before launch carries its predecessor's runner pid and start time. The "never started" check required a null pid, so this run waited out the closure timeout, held the board's Retry for ten minutes, then settled as unknown unpriced usage and paused the company. The task said "budget" although almost nothing was spent. > - This pull request recognizes an identity that the run only inherited, so the run settles at once with no charge. > - The benefit is that a pre-launch failure after a restart no longer blocks Retry or pauses the company. ## Linked Issues or Issue Description What happened? In restart-chaos wave seed 9802 (local, G1), task t02 graded R3 `retry_no_new_run`. Follow-up run 0b4b404f failed 0.3 s after start with `native_session_cleanup_quarantined`. It recorded no `native.process_start_requested`, but its row carried `process_pid` 11356 and the start time of predecessor run 71f03e16 in the same native session. Its accounting stayed pending until 11:37:23, ten minutes later, then settled as an `unpriced` cost event with billing type `unknown`. The board Retry, accepted at 11:28:37 and held until that accounting released, was then refused with "Company cannot start work because recorded usage has no reliable price". The company was paused with `pause_reason = budget`, and the task showed `userAction: "budget"` with no working Retry. Expected behavior A native run that never launched its own runner settles at once with no charge, whatever runner identity it inherited. The held Retry dispatches, and the company keeps running. Steps to reproduce 1. Have a native follow-up run inherit its predecessor's runner identity (pid and start time). 2. Fail it before it records any process start (for example, with a quarantined cleanup). 3. Enable a company budget policy that blocks unpriced usage, then run accounting. The run stays pending until the closure timeout, settles as unknown unpriced usage and pauses the company. Paperclip version or commit master at `0d80b74c21` ## What Changed - `server/src/services/heartbeat/runtime-selection.ts`: when selection copies a previous run's process identity into a native run that has none of its own, it also records `inheritedProcessIdentityRunId` in that run's `runnerProfileJson`. - `server/src/services/run-cost-accounting.ts`: `nativeProviderWorkNeverStarted` accepts either a null process identity, or the `inheritedProcessIdentityRunId` marker with no own `native.process_start_requested` / `native.process_identity_recorded` event. It never reads the predecessor's mutable process columns, so settled-process cleanup cannot erase the proof and no session-history scan is needed. - Tests: - `runtime-selection.test.ts`: selection writes the marker only when a prior identity is copied. End to end, after cleanup clears the predecessor's process columns, `accountRunCost` settles the refused follow-up at once with no cost event. - `closed-native-receipt-accounting.test.ts`: the marker settles at once with no charge and no company pause under a blocking policy. An own recorded start, or an identity without the marker, stays pending. ## Verification - Red before the fix: the inherited case stays pending (`accountRunCost` returns `false`). - `runtime-selection` (except six dynamic-skill tests that need a local `.claude/skills/slack/SKILL.md`, which fail identically on master), `closed-native-receipt-accounting`, `cost-data-quality`, `admission-independent-of-cleanup`: pass. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server`: passes. ## Risks - Low. The predicate widens only for a run that carries the selection-written marker and recorded no process start of its own. Receipt sequence, fences, journaled receipts and a settled coordinator are still required. - Previous-release story: forward-only. Runs selected by an older build have no marker. Their pending debt keeps today's behavior and settles at the accounting closure timeout. paperclipai/paperclip#20254 (budget owner) makes such a lost-receipt charge no longer trip a blocking unpriced-usage policy, so those runs no longer pause a company. No schema change. - Out of scope: the session cleanup quarantine itself (paperclipai/paperclip#20260). ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:12:01 -06:00
DottaandClaude Opus 5.5	4c15eea770	fix(accounting): a run's unproven accounting never holds the company (#20254 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - After a restart, each lost provider receipt leaves an accounting debt that settles later > - In restart-chaos G1 (seed 9802, task t02), six debts each waited the full 10-minute closure bound, and the board's accepted Retry waited with them > - The last debt settled as an unpriced charge. Under the blocking unpriced-usage policy, that charge hard-stopped the whole company, which cancelled the Retry, and the task showed "budget" > - Dotta's plan decision 1 says accounting is never a company-wide gate: admission reads per-run exposure and real hard stops only > - This pull request makes those debts settle at once when the stop is proven, records accounting-settled charges without a company hold, and labels the remaining accounting wait correctly > - The benefit is that a restart no longer stalls a company's work for 10 minutes or stops it outright ## Linked Issues or Issue Description Refs #20250 (Phase 2: a run that only inherited its predecessor's runner identity settles at once; this PR builds on it), #19944, #19826, #19857 What happened? Restart-chaos wave w-20261109T111954-s9802 (G1), task t02, master `c5da0b88ad`. After a K-fault restart at 11:27, six `run_accounting_debt` records were "unprovable". Each settled exactly 10 minutes after its run finished: bf9ea6b4 and c2e222a2 at 11:30, 097d0036 at 11:35, f64f9f46 and aa2c9f03 at 11:36, and 0b4b404f at 11:37:23. - Four runs were `cancelled` (issue_terminalized), with `native.local_process_stopped` as their last process event and no recorded pid. The receipt-closed rule accepted that stop proof only for `failed` runs. - 0b4b404f failed with `native_session_cleanup_quarantined` before it started any process. It journaled no receipt, but it carried pid 11356 of the previous run's warm runner, so the never-started rule did not apply. - At 11:37:23, 0b4b404f settled as an unpriced unknown-billing charge with zero tokens. With `unpricedUsagePolicy: block`, the company hit a hard stop at 11:37:24 (24 of 3000 cents). The pause cancelled the board's Retry (751751ea, accepted at 11:28:37), and the task showed `userAction: "budget"`. Expected behavior Accounting debts whose provider stop is proven settle at once. Accounting never holds or pauses the whole company. A wait caused by unpriced usage is labeled as accounting, not budget. Steps to reproduce Restart the control plane while native runs are being cancelled, with the company's unpriced-usage policy set to block. Before this change, the debts wait 10 minutes and then hard-stop the company. ## What Changed - `run-cost-accounting.ts`: a verified local process stop closes the receipt source for `cancelled`, `timed_out` and `interrupted` runs as well as `failed` ones. The inherited-pid run (0b4b404f) is covered by #20250, so this PR does not change the never-started rule. - `budget-exposure.ts`: a final charge that accounting settled without its receipt (receipt unavailable, closure timed out, capture failed, or lost on restore) never trips the blocking unpriced-usage policy. Unpriced usage from a complete receipt still does. A capture failure stays exempt even if a later report adds usage, because its receipt is never certified complete. This removes the restart-derived special case, because the rule now covers every accounting-settled charge. - `task-interruption-presentation.ts`: a `budget.blocked` wait caused by the unpriced-usage refusal is shown as `accounting.unpriced_usage`, with accounting wording, for both a held Retry and a never-started task. Only the fixed refusal phrase is matched; the stored message is never shown. - `doc/SPEC-implementation.md`: the admission and accounting rules are updated to match. ## Verification - `closed-native-receipt-accounting.test.ts`: new tests for cancelled, timed-out and interrupted runs with a verified stop; they were red before the fix. When a timed-out closure settles under the blocking policy, the board's saved Retry is promoted (due at once) and never cancelled. That assertion was red on master, where the hard stop cancelled it. Seven tests that asserted the old rule (an accounting-settled charge keeps the blocking policy) now assert that the charge is recorded and the company stays active and admits work. All 61 pass. - `task-interruption-query.test.ts` (Postgres): a Retry held by the unpriced refusal, and a skipped wake, are labeled `accounting.unpriced_usage`. A budget hard stop keeps `budget.blocked`. The test was red before the fix. - Also passing: `run-cost-accounting`, `cost-data-quality`, `idle-accounting-work`, `native-usage-delivery.integration`, `task-interruption-presentation`, and the accounting and budget tests in `heartbeat-process-recovery`. Server typecheck has no errors in the changed files. The only local errors are in `native-session-executor.ts`, from a stale local runner build. - Not run: the G1 chaos rerun. Phase 3 runs it after merge. ## Risks - Behavior change: a charge that accounting settled without a receipt no longer hard-stops a company under the blocking policy. That includes a remote run whose closure evidence never arrived, which may still have spent. The charge stays in Costs with provenance and an activity entry, and the spend still counts toward the budget limit. This follows plan decision 1. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test execution). Extended reasoning enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:02:47 -06:00
DottaandClaude Opus 5.5	7044ff96e3	fix(api-catalog): mark task-tree controls board-only and name the Pause subtree page (#20252 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents read the API catalog (`server/src/routes/openapi.ts`) to find what they can call and who performs each action > - A task-tree pause holds a parent task and all of its subtasks. Only a person can apply it; every tree-control route calls `assertBoard` > - The catalog did not list these routes as board-only, so it showed them as `board_or_agent`, and it did not say where a person pauses a tree > - A chat agent asked to "pause all work on the launch until tomorrow" did not use or name the tree pause. It moved the running child tasks to backlog, cleared their assignees, and told the person that all work was paused > - This pull request marks the six tree-control routes board-only and describes the human control on the task page > - The benefit is that the agent can send the person to the real persistent pause instead of improvising one ## Linked Issues or Issue Description What happened In the chat eval `steering.stop-versus-pause` (runner-opencode, GitHub Actions run 42606822088), the CEO agent got "Pause all work on the launch until tomorrow." It sent `PATCH /api/issues/{id}` with a backlog status for the two running Website tasks. That cleared their execution assignments. It replied "All launch work is paused." In a sibling eval run where the agent called `POST /api/issues/{id}/tree-holds`, it got 403 "Board access required". The catalog showed that route as callable by agents. Expected behavior The catalog marks every tree-control route board-only. The create-hold entry tells the agent that a person pauses the tree on the task page with More task actions, then Pause subtree, and resumes it there. It also says that backlog or reassignment is not a tree pause. Steps to reproduce 1. Run `buildOpenApiSpec()` on master. 2. Read `paths["/api/issues/{id}/tree-holds"].post["x-paperclip-authorization"]`. 3. It is `{ actor: "board_or_agent" }`, but the route calls `assertBoard`. ## What Changed - Added the six tree-control operations (`tree-control/state`, `tree-control/preview`, `tree-holds` list/create/get, `tree-holds/{holdId}/release`) to `BOARD_ONLY_OPERATIONS`. - Added a description to `POST /api/issues/{id}/tree-holds` that names the board-only rule, the task page path, the Pause subtree and Resume subtree controls, and says backlog or reassignment is not a tree pause. - Added an `openapi-routes.test.ts` case for the board-only authorization and the description. ## Verification - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts -t "task-tree controls"` fails before the change and passes after it. - `server/src/__tests__/openapi-routes.test.ts` and `server/src/services/native-runtime/runner-api.test.ts` pass (2111 tests). - The fix is cherry-picked onto the chat-eval branch for an end-to-end rerun of `steering.stop-versus-pause` and `task-coordination.pause-tree`. ## Risks - Low risk. Catalog metadata only. The routes already reject agents, so no runtime permission changes. - Agents now see these operations as board-only. That matches the existing 403. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 06:56:53 -06:00
DottaandClaude Opus 5.5	ba75b44c3c	fix(chat): name the addressee on human-only card denials and list team requirements (#20245 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat answers a person's requests with a chat agent that reads API results and the chat guidance > - A decision card can be human-only and addressed to one person, and the teams catalog lists ready-made teams with their requirements > - When an agent tried to accept another person's card, the denial said only "human-only", so the agent told the asker to accept the card themselves > - When an agent recommended a catalog team, the guidance asked only for roles, so the agent left out the required skills > - This pull request makes the denial name the addressee as the only resolver, and asks agents to list a team's roles, required skills and configuration inputs > - The benefit is that chat answers send people to the right person and show what a team needs before install ## Linked Issues or Issue Description Refs #20206, #20220, #20239 What happened Alex wrote "I approve Riley's request." in Agent Chat. The pending card is human-only and addressed to Riley. The agent tried to record the decision and got `403 interaction_human_only` with only "This issue-thread interaction is human-only". The reply correctly said the card stays pending, but it then told Alex to "accept or decline that request" on the task. Alex cannot do that; only Riley can. In a second chat, a person asked "Do we have a ready-made team for building a website?". The agent recommended Product Engineering and its roles. It did not name the four required skills that the catalog returned in `requiredSkills`. Expected behavior When an agent cannot resolve a card that is addressed to another person, the reply names that person as the only one who can resolve it. The reply does not tell the asker to act on the card. When an agent recommends a catalog team, the reply names the team's roles, every required skill and every configuration input. It says that a catalog listing does not mean the accounts are configured. Steps to reproduce 1. As user Alex, create a `request_confirmation` card with `resolverPolicy: "human_only"` and `addresseeUserId` set to user Riley on Alex's chat task. 2. In that chat, write "I approve Riley's request." 3. Read the agent run. `resolve-from-comment` returns 403 `interaction_human_only` without the addressee. The reply tells Alex to accept or decline the card. 4. In a new chat, ask "Do we have a ready-made team for building a website?". 5. Read the reply. It names Product Engineering and its roles, but not its required skills. ## What Changed - `server/src/services/issue-thread-interaction-resolution.ts`: when an agent is denied on a human-only card that is addressed to a person other than the verified replying user, the denial says the card is addressed to that person, that only they can accept or decline it, that the denial is final, and that the agent must not ask the asker to act on the card. The details include `addresseeUserId`. Cards without an addressee keep the old message. - `server/src/routes/issues.ts`: the route adds the addressee's display name to that denial (`details.addresseeName` and the message). If the name lookup fails, the generic addressee text stays. - `server/src/services/agent-conversations.ts`: the teams catalog paragraph asks agents to name a recommended team's roles (`agentSlugs`), every required skill (`requiredSkills`, by ref) and every configuration input (`envInputs`), and to say that a listing does not mean accounts are configured. - Tests: evaluator unit tests, a route-level test for the named denial, and a directive assertion. ## Verification - `cd server && npx vitest run src/services/issue-thread-interaction-resolution.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/issue-thread-interaction-routes.test.ts src/__tests__/confirmation-comment-resolution.test.ts`: 4 files, all pass. - `tsc --noEmit -p server/tsconfig.json`: passes. - The change is cherry-picked onto the eval branch `chat-eval/case-reply-completeness`. The live Runner E2E cells `questions-and-approvals.wrong-reviewer` and `teams.find-team` run there. ## Risks - Low risk. The resolver decision does not change: the same requests are allowed and denied. Only the denial text and details change for agents on addressed human-only cards. - The denial reads one user row for the display name. A failed read falls back to the generic text. - The chat directive is slightly longer, so each chat prompt uses a few more tokens. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 06:48:54 -06:00
DottaandClaude Opus 5.5	3374ca54f5	fix(runner): record a runner exit mid-turn as runner loss, and stop the lineage sampler panicking (#20259 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The native runner (`runnerd`) runs each provider turn, and the server records why a run stopped and what the person can do next > - When `runnerd` exits during a provider turn, the server saved the run as `native_provider_terminal_failed`: "The provider session is permanently unusable", with automatic retries stopped > - The real cause was the runner process, and the transport already names it (`native_runner_process_exited`) at the start of the closed-stream message > - Separately, the process-lineage sampler added in #19727 panicked on Linux debug builds when a provider tool started its own session, and that panic text became the run's visible failure > - This pull request records a runner exit mid-turn as runner loss, and reads session IDs without the assertion that panicked > - The benefit is that a person sees the true cause of a stopped run, and recovery treats a lost runner like every other runner exit ## Linked Issues or Issue Description Refs #19727 What happened A chat eval killed only the owned `runnerd` process (SIGKILL) while a provider tool was running. The run failed with `errorCode: native_provider_terminal_failed` and the next action "The provider session is permanently unusable". The error text was `Notification transport failed closed: native_runner_process_exited: runnerd exited unexpectedly: paperclip-runnerd panic: panicked at rustix-1.1.4/src/pid.rs:61:9: assertion failed: raw > 0`. The panic came from the lineage sampler thread before the kill. It called `rustix::process::getsid`, which asserts a nonzero session. A kernel thread, or a process whose session leader is outside the PID namespace, reports session 0. Expected behavior A runner process exit is recorded as `native_runner_process_exited`, the same cause the server already uses for a runner exit outside a turn. Recovery then applies its normal runner-loss rules (checkpoint evidence, bounded attempts, cleanup quarantine). The lineage sampler does not panic, and its diagnostics do not replace the real failure text. Steps to reproduce 1. Start a native OpenCode run on Linux with a debug `runnerd`. 2. Let the agent run a shell tool that starts its own session (for example a long `node -e` command). 3. Send SIGKILL to the `runnerd` PID only. 4. Read `GET /api/heartbeat-runs/<runId>`: `errorCode` is `native_provider_terminal_failed` and the error names the rustix panic. ## What Changed - `nativeSessionFailureSourceCode` returns `native_runner_process_exited` for a `NativeProviderTerminalFailure` with code `notification_transport_failed` when its message starts with `Notification transport failed closed: native_runner_process_exited: `. Only the runner transport writes that prefix. Other closed-stream causes stay `native_provider_terminal_failed`. - `paperclip-process-table` has a new `session(pid)` function. Linux reads the session field of `/proc/<pid>/stat`. macOS calls `getsid`. A session of 0 or an error gives `None`. - `process_lineage` reads sessions through `paperclip_process_table::session` instead of rustix `getsid`. - Regression tests: the runner-loss classification (attempts 1 and 3, and three negative shapes), the stat session parser with a kernel-thread line, and a live read of this process's session. ## Verification - `cd server && npx vitest run src/services/native-runtime/native-session-executor.test.ts -t "runner process exit that closed\|preserves stable provider and runner\|model rejection from a provider terminal\|operator-owned through terminal"` (8 passed) - `cd packages/paperclip-runner/runner && cargo test -p paperclip-process-table` (macOS: 3 passed; the Linux-only parser tests run in CI) - `cd packages/paperclip-runner/runner && cargo test -p paperclip-runner-core --lib process_lineage` (6 passed) - End to end: the `recovery.*` chat eval cells that kill `runnerd` are run against this change on a separate eval branch. ## Risks - Behavior shift: a runner exit during a turn now follows runner-loss recovery instead of stopping as a permanent provider failure. Recovery still decides from durable evidence. Ambiguous state stays terminal, and an unverified cleanup stays quarantined. - Previous-release story: no persisted shape changes. Runs saved earlier with `native_provider_terminal_failed` keep that code. Only new failures are classified with the new rule. - The session read on Linux now uses `/proc` instead of a syscall. It gives the same value, and it reports 0 (now `None`) in the same cases. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 06:38:24 -06:00
DottaandClaude Opus 5.5	b3bf12a5db	fix(workspaces): close a retained local task directory as archived, not cleanup_failed (#20248 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A task runs in an execution workspace. A person or an agent can close that workspace and keep its files for later > - A local workspace directory that the runtime did not create is never removed on close. Examples are an operator directory and an isolated task directory that retention keeps > - The cleanup result still said `cleaned: false`, because that directory still exists > - So every close of such a workspace got the status `cleanup_failed`, with no cleanup reason > - This pull request treats that directory as retained by design, so the close records `archived` > - The benefit is that a normal close no longer shows a failure that did not happen ## Linked Issues or Issue Description Refs #16776 (related: records-only archive of shared non-git sessions; that PR changes close readiness, this PR changes the recorded close outcome) What happened A chat agent closed a completed task's execution workspace with `PATCH /api/execution-workspaces/:id` and `status: "archived"`. The workspace is a `local_fs` isolated task directory with `createdByRuntime: false`. The workspace got `status: "cleanup_failed"`, a `closedAt`, and `cleanupReason: null`. The files stayed, as intended. The agent then had to tell the person that cleanup failed. Expected behavior The close records `status: "archived"`. The retained directory is the intended outcome, so it is not a failed cleanup. A runtime-created directory is still removed. Steps to reproduce 1. Create a task workspace whose `local_fs` directory exists and has `metadata.createdByRuntime: false` (for example an isolated task directory). 2. Close it with `PATCH /api/execution-workspaces/:id` and body `{"status":"archived"}`. 3. Read the workspace. Its status is `cleanup_failed` and `cleanupReason` is `null`. ## What Changed - `cleanupExecutionWorkspaceArtifacts` reports `cleaned: true` for a `local_fs` directory that the runtime did not create. It never removes that directory, so nothing is left to clean. - Added `server/src/__tests__/workspace-cleanup-retained-local-dir.test.ts`: a retained directory reports `cleaned: true` and keeps its files; a runtime-created directory is still removed. ## Verification - `pnpm exec vitest run server/src/__tests__/workspace-cleanup-retained-local-dir.test.ts` fails before the change (`cleaned: false`) and passes after it. - `server/src/__tests__/workspace-runtime.test.ts` (179), `execution-workspaces-routes.test.ts` (17), `execution-workspaces-service.test.ts` (85) and `workspace-delivery-cleanup.integration.test.ts` (19) pass. - End to end: the chat eval `workspaces.close-and-reopen` (runner-opencode) failed only on `status: cleanup_failed` after a correct agent close in GitHub Actions run 42606822088. This fix is cherry-picked onto the eval branch for a rerun. ## Risks - Low risk. Only `local_fs` workspaces with `createdByRuntime` not `true` change. Git worktrees, runtime-created directories, plugin and remote workspaces keep their behavior. - The terminal reaper uses the same cleanup function. It now records `archived` instead of `cleanup_failed` for such a retained directory. Isolated task directory retention treats both statuses as closed, so retention is not changed. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 06:25:15 -06:00
DottaandClaude Opus 5.5	7adb78d811	perf(heartbeat): retry a preparation a restart interrupted at once (#20255 ) ## Thinking Path - Restart recovery should resume interrupted work quickly. #20072 added an `interrupted_preparation` retry lane for runs whose preparation a restart interrupted before any provider work. - Phase 3's local R1 replay (`w-20261109T110754-s9807`) showed `restart_recovery_control_plane` p95 about 41 s for both Claude and Codex (n=6). - From the kept database, the earlier slow samples are back-to-back faults (#20243 now skips those). The two that remain uninterrupted, Codex `8a008160` and Claude `ca74c302`, are `interrupted_preparation` successors: - created at 11:09:41.7–41.8, as f03 recovered; - `scheduled_retry_at` 26.3 s and 34.2 s later; - claimed 18.5 s and 26.5 s after f04 was ready. - The lane reused the transient-failure backoff (30 s, then 60 s, jittered). A restart isn't a transient fault, so the successor shouldn't wait. ## Linked Issues or Issue Description Refs #20072 What happened? After a restart interrupted a run's preparation, its successor was scheduled with the transient-failure backoff (about 30 s). Recovery took about 40–50 s instead of a few seconds. Steps to reproduce 1. Interrupt a run's preparation with a control-plane restart (for example K1/K2 during workspace preparation). 2. The successor is created with `scheduled_retry_reason: interrupted_preparation` and `scheduled_retry_at` about 30 s out. Expected behavior The first retry of a preparation a restart interrupted starts at once. Backoff applies only if it keeps being interrupted. ## What Changed - `server/src/services/heartbeat/retries.ts`: new `computeInterruptedPreparationRetrySchedule`, used by the interrupted-preparation lane. - Attempt 1 is due immediately (delay 0). - Attempt 2 onwards uses the transient schedule, shifted by one, with the longest delay from then on. - The lane's attempt cap (`INTERRUPTED_PREPARATION_RETRY_MAX_ATTEMPTS`) and the failure-budget guard are unchanged. - `retries.test.ts`: a schedule test covering an immediate first attempt, backoff afterwards, the longest delay later, and invalid attempts. It's red on master. ## Verification - `retries.test.ts`: 24 passed. - `interrupted-preparation-hold-settle`, `workspace-recovery-preparation-dispatch` and `run-dispatch/adapters/postgres` tests: 118 passed. - `heartbeat-process-recovery.test.ts` filtered to interrupt, prepare and retry cases: 59 passed and 1 failed. The failure, "escalates (does not retry) an accepted-interaction-continuation setup failure…", is order-dependent within the filtered batch: it passes alone both on master and with this change. The unfiltered file exceeds the local timeout; CI runs it. ## Risks - A preparation that itself crashes the server now retries once immediately before backing off. It stays bounded by the lane's attempt cap and the existing failure-budget guard. - No change to transient-failure retries or to other lanes. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`, Anthropic), 1M context window, tool use and code execution, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 06:22:06 -06:00
DottaandClaude Opus 5.5	2ea1a4f4eb	fix(chat): tell chat agents what a status card is and to reply now after queuing one (#20230 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat lets a person talk to an agent in a standing conversation > - Status Cards is an experimental feature: a saved card on the company's Status page that a summarizer agent keeps up to date > - A chat agent cannot read the experimental settings, so nothing it reads says that status cards exist, how to create one, or where a card lives > - When a person asks for a status card, the agent writes a Markdown task document instead. When a person asks for a refresh, the agent queues it and then sleeps and polls the card for minutes > - This pull request tells the chat agent about Status Cards only while the experiment is on, and makes the create and refresh catalog entries say that the work is queued and the agent replies now with the card link > - The benefit is that the agent creates a real card, links it, and does not hold the chat open while the summarizer works ## Linked Issues or Issue Description What happened With Status Cards enabled, a person asked the CEO agent "Make me a status card showing the launch's biggest blockers." The agent's reasoning treated "status card" as a format. It never searched for the feature and saved a Markdown task document. In a second chat, a person asked "Refresh the launch status card now." The agent searched for about 54 s to learn what a status card is, called the refresh route, and then slept 20, 30 and 45 s to poll the card, although its guidance says not to sleep or poll. The refresh response also did not include the card's page link. Separately, a teammate assigned as the card's summarizer spent over two minutes looking for the undefined summary sentinel format named in its update task, so a failed generation never reached its blocked state in time. Expected behavior While the experiment is on, the agent knows that a status card is a saved feature at `/<prefix>/status/<cardId>`. It creates one with `POST /api/companies/{companyId}/status-cards`, or refreshes one with `POST /api/status-cards/{id}/refresh`. It replies at once with the card as a Markdown link and says that the summarizer updates the card. When the experiment is off, the agent sees no status card guidance. Steps to reproduce 1. Enable Status Cards in Instance Settings > Experimental. 2. Open a chat with the CEO agent. 3. Send "Make me a status card showing the launch's biggest blockers." 4. See that the agent writes a task document and no card appears on the Status page. ## What Changed - Add `STATUS_CARDS_CHAT_DIRECTIVE` in `server/src/services/agent-conversations.ts`. It says what a status card is, names the Status page and card routes, names the create and refresh routes, and tells the agent to reply now with the card's `href` as a Markdown link instead of polling. - Render the directive in standing-chat task markdown only when `statusCardsEnabled` is true (`server/src/services/heartbeat/task-markdown.ts`). Set the flag from the experimental settings that the heartbeat already loads (`server/src/services/heartbeat/task-context.ts`). No new query runs before the turn starts. - Add descriptions to the create and refresh entries in the API catalog (`server/src/routes/openapi.ts`). They say that the summarizer does the work in its own run, and that the agent replies now with the returned link. - Return the card's `href` in the refresh response and in the card list, as create, get, update and archive already do (`server/src/routes/status-cards.ts`). An agent that reuses a listed card can then link it. - Tell the agent to report the outcome the refresh response gives: queued, already running, or no run started (deferred by policy, or a concurrent refresh already finished). - Define the summary sentinels in the update task prompt (`server/src/services/status-cards.ts`). The prompt told any assigned summarizer to "preserve" the built-in Summarizer's `STATUS:` and `<<<SUMMARY-DRAFT>>>` sentinels without defining them. A non-built-in summarizer searched for the format for minutes. The prompt now gives the exact lines, says the PUT body is plain Markdown, and says to comment and block the task when no summary can be produced, so the card keeps its previous summary and shows the failure. - Add tests: the directive appears only in standing chats with the experiment on, and its page paths are real routes in `ui/src/App.tsx`; the catalog descriptions; the refresh response `href`. ## Verification - `cd server && npx vitest run src/services/heartbeat/task-markdown.test.ts src/__tests__/openapi-routes.test.ts src/__tests__/status-cards.test.ts src/__tests__/agent-conversations.test.ts` passes (all pass; `status-cards.test.ts` also checks the update task prompt). - `npx tsc --noEmit -p server` reports no errors in the changed files. - End to end: the `status-cards.create-card` and `status-cards.refresh-card` chat evals run with this commit cherry-picked onto their case branch. ## Risks - Low risk. The text is added only to standing-chat context and only while the experiment is on. - The refresh response gains one field (`card.href`). Existing clients ignore unknown fields. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 05:39:01 -06:00
DottaandClaude Opus 5.5	9db7da444f	fix(chat): point agents at the teams catalog for ready-made teams (#20239 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat answers a person's requests with a chat agent that reads the API catalog and the chat guidance > - Paperclip ships a teams catalog of ready-made teams that a company can preview and install > - The chat guidance does not name this catalog, and its API catalog entries have only short summaries > - So agents asked for a "ready-made team" check only the existing agents and skills, or search npm and GitHub > - This pull request names the teams catalog where the agent already looks: the chat guidance and the API catalog entries > - The benefit is that agents find, preview and link ready-made teams instead of saying none exist ## Linked Issues or Issue Description What happened An agent was asked "Do we have a ready-made team for building a website?". It read only the company's agents, projects and skills. It replied that the company has no website team and offered to hire agents. It did not read the teams catalog, which has a Product Engineering team. An agent was asked "Show me what installing the website team would add.". It found `GET /api/teams/catalog`, but found no team named "website". It then searched local files, the skills catalog, npm and GitHub until the turn timed out. It did not preview the install. Expected behavior The agent reads the teams catalog and chooses a team by its description and roles. It previews an install with the preview route, which creates no resources. It does not link a teams catalog page, because the web app does not route one. It does not search outside Paperclip for other teams. It knows who can install a team, and it names the `paperclipai teams install` command for a board user. Preview also works in Ask and Plan modes, because it saves no data. Steps to reproduce 1. Create a company with only a CEO agent. 2. In Agent Chat, ask "Do we have a ready-made team for building a website?". 3. Read the run. The agent does not call `GET /api/teams/catalog`, and the reply says no team exists. 4. In a new chat, ask "Show me what installing the website team would add.". 5. Read the run. The agent searches npm and GitHub and does not call the preview route. ## What Changed - `server/src/services/agent-conversations.ts`: the chat directive has a new short paragraph. It says that ready-made teams are packages in the teams catalog. It tells the agent to list the catalog, to choose by description and roles, and not to search package registries, repositories or the web. It names the preview route and says that preview creates no resources. It says who can install. It says that the web app has no teams catalog page, so the agent must not link one, and it names the `paperclipai teams install <team id> --company-id <companyId>` command for a board user. - `server/src/routes/openapi.ts`: `GET /api/teams/catalog` now has a description and documents its `q`, `kind` and `category` query parameters. The preview route says that it creates no resources. The install route says who can install and names the CLI command. - `server/src/services/native-runtime/runner-api-catalog.ts`: the team preview route is allowed in Ask and Plan modes, like other reads. It computes a plan and saves no data. Install stays in standard mode only. - Tests in `agent-conversations.test.ts` and `runner-api.test.ts`. The API search for "ready-made team" now returns `GET /api/teams/catalog` first. A test validates a preview call in Ask mode and rejects install in Ask mode. ## Verification - `pnpm exec vitest run --root server src/services/native-runtime/runner-api.test.ts src/__tests__/agent-conversations.test.ts src/__tests__/openapi-routes.test.ts src/__tests__/teams-catalog-routes.test.ts`: all tests pass. - The change is also cherry-picked onto the eval branch `chat-eval/case-teams`. The live Runner E2E cells for `teams.find-team`, `teams.preview-install` and `teams.install-team` run there. ## Risks - Low risk. Most of the change is text in the API catalog and in the chat guidance, plus documented query parameters that the route already reads. - One behavior change: agents in Ask and Plan modes can now call the team preview route. The route only computes a plan, and it applies the same company access check. - The chat directive is slightly longer, so each chat prompt uses a few more tokens. - Related open PR #18970 adds request body schemas to the same team routes. The two changes touch nearby lines and can need a small merge. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 05:36:24 -06:00
DottaandClaude Opus 5.5	3f2a940d5b	fix(runner): answer held Paperclip tool calls after a local restart before the next turn (#20227 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a restart interrupts a turn, the Runner keeps each Paperclip tool call the model made as "held" until Paperclip answers it, and it refuses to start the next turn while any call is held > - Paperclip answers held calls after a reattach through its tool recovery path. That path was wired only for remote runners > - In restart chaos wave `w-20261109T084023-s9403`, t101 (local codex, K2 then K1) held a `read_document` call that nothing answered, so the continuation's `turn.start` failed and the task needed a Retry > - This pull request wires held-call recovery for local runners too, and makes `turn.start` answer or wait for held calls first > - The benefit is that a turn lost to a restart no longer strands the run behind one unanswered call ## Linked Issues or Issue Description Refs #19831. Restart chaos wave `w-20261109T084023-s9403` (master `fc5c79a327`), task t101 (codex C1, K2, local). What happened: - Run `4b5810d5` called `read_document` (`exec-ac8fa9b4`) around the K2 drain at 09:08:18. A K1 then killed the next server start. - At 09:08:36 the third controller reattached. The Runner restored the conversation and reported the old turn lost (`provider_turn_lost_on_restore`, recoverable). The held call stayed pending, by design: a held call accepts a late result. - The controller had the call: the quarantine detail lists `exec-ac8fa9b4` under `semanticTools.pending`. But `recoverPendingSemanticInputs()` returned at once: for a local run `controllerPersistence` is unset, so `recoverSemanticToolInput` was `undefined`. - At 09:08:46 the continuation's `turn.start` failed with "cannot prepare the next provider turn while tool calls are active (pending: 1 [read_document:exec-ac8fa9b4…])". The run ended `native_session_interrupted`. ## What Changed - `server/src/services/native-runtime/native-session-executor.ts`: the runnerd transport now always gets `recoverSemanticToolInput`, not only when `controllerPersistence` (remote) is set. It uses the same epoch check and `SessionAuthorityEpoch.recover`. Recovery re-authorizes and re-runs reads, and replays receipts for mutations; it never repeats an uncertain mutation. - `packages/paperclip-runner/src/live/runnerd-codex-transport.ts` (`#startTurnWork`): when the control plane has unsettled semantic inputs, it runs `recoverPendingSemanticInputs()` before queueing `turn.start`, and waits for that pass, bounded by the turn-start deadline. Recovered results are queued as commands ahead of `turn.start`, and the Runner applies commands in order. If recovery has not finished by the deadline, the turn start fails with `native_turn_start_held_tool_calls_unresolved` instead of sending `turn.start` behind it. When everything is settled there is no wait, so a normal turn start is unchanged. Live calls of a turn being replaced are not waited on; the replaced start still rejects them. - `packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts`: `recoverPendingSemanticInputs()` passes are serialized. A call made while the startup pass is still answering a held call waits for that pass, then runs its own, so the turn-start gate cannot finish early by skipping a call that is already being recovered. ## Verification Red/green: - `native-session-executor.test.ts` "answers a local run's held Paperclip tool calls after a restart (restart chaos t101)". On master `recoverSemanticToolInput` is `undefined` for a local run; with this PR it is wired. - `runnerd-codex-transport.test.ts` "answers a held call before starting the next turn (restart chaos t101)", against real runnerd and fake Codex, with a slow startup recovery pass already in flight when `turn/start` arrives. On master `turn/start` never ran recovery, and without the serialization `turn.start` is queued before the held call's result. With this PR the held call is recovered once and its result is queued before `turn.start`. Regression and static checks: - `runnerd-codex-transport.test.ts`, `durable-prp-control-plane.test.ts`, `runnerd-codex-transport-adoption.test.ts`: 510 passed. - `native-session-executor.test.ts` and `native-session-resume.test.ts`: 1368 passed, 1 failed. The failing case, "archives a damaged prior epoch … with real runnerd", fails identically on master on this Mac (`ENOENT … authority-epochs`), so CI is its reference. - `tsc --noEmit` passes for `packages/paperclip-runner` and `server`. ## Risks - Local runs now recover held calls after a reattach, as remote runs already did. Reads re-run under current authority. A mutation without a saved receipt still returns no result and stays held, as before. - `turn.start` may now wait up to its existing deadline while a recovery pass runs, then fails clearly if it has not finished. It never waits when nothing is unsettled, so normal turns get no added latency before `turn.start`. - No schema change and no change to persisted run state formats. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 05:26:12 -06:00
DottaandClaude Opus 5.5	c5da0b88ad	fix(native): take over a remote runner at once when its old controller is proven gone (#20240 ) ## Thinking Path - Restart recovery should be fast and fault-proof; "restarts are product failures". Remote (Daytona) runs must recover as quickly as local ones once it is safe. - Phase 3's Daytona K13 storm (`w-20261109T100435-s9503`) showed Codex `restart_recovery_control_plane` p50 34.9 s, p95 61.1 s. That's Paperclip's own share, before any provider load. - The run logs show why. After a hard restart, each slow run sat in "Native restart recovery is waiting for safe ownership evidence" with reason `remote_controller_lease_live` for about 51 s, until the dead controller's 60 s remote-controller lease expired. - On the remote path, `evaluateNativeControllerTakeover` deliberately skips every process proof and waits for the lease. That's correct when the previous controller might live in another container or on another host. When it recorded this exact process space, the shared prover (pid, process start, boot id) can prove it gone, which is the same proof the local path already trusts. ## Linked Issues or Issue Description What happened? After a hard control-plane restart, a remote (Daytona) native run could not be recovered until the dead controller's 60 s remote lease expired. Recovery took about 35–60 s, even though the old controller process was provably gone. Steps to reproduce 1. Run a Codex task in a Daytona environment. 2. Hard-kill the control plane (SIGKILL) while the run is active, then restart it. 3. The run logs `native.recovery.transition` "waiting for safe ownership evidence" (`remote_controller_lease_live`) and resumes only when the lease expires. Expected behavior When the previous controller ran in this same process space and is proven gone, recovery takes over the remote runner immediately. It still waits for the lease whenever the previous controller cannot be proven gone. ## What Changed - `server/src/services/native-runtime/native-restart-recovery.ts`, `evaluateNativeControllerTakeover` with `remoteLeaseFenced`: - An expired or released lease is allowed, as before. - A live lease whose holder recorded this exact process space (`controllerHostHash`, neither missing nor unknown) and a controller PID goes through the shared owner-gone prover, with `requireStoppedController`. A positive proof allows the takeover, with reason `remote_lease_<proof reason>`, for example `remote_lease_controller_process_dead`. - The one-release null-host fallback is never accepted for a remote runner. - A holder in another process space is never probed (the existing "must not probe another container's PID" test still holds). It waits for the lease, as before. - `native-restart-recovery.test.ts`: a new test, red on master, covers: - takeover when this-process-space controller is dead; - waiting while it is alive; - waiting without a probe when the process space differs. ## Verification - The 7 server test files that cover the takeover decision pass (351 tests): `native-restart-recovery`, `native-controller-host-recovery`, `native-remote-retry-admission`, `native-recovered-owner-followup`, `native-runner-restart-diskless.integration`, `heartbeat/recovery` and `controller-instance-lease`. - Evidence: kept-instance DB copy from the Daytona K13 wave. In run `ba194ae7`, the `awaiting_evidence` transition `remote_controller_lease_live` lasted from 04:26:06.8 to 04:26:57.9, then `bootstrap_incomplete` restart. - A Daytona K13 rerun will measure the before and after. ## Risks - Takeover soundness rests on the shared prover, which the local path already uses for the same controllers: same process space, PID dead or definitely restarted, and boot-id proof where needed. If the old controller is alive, the decision still waits for the lease. - Multi-host or containerized controllers record a different process space, so they are unaffected and still fenced by the lease. - The remote runner keeps its own reconnect grace (7 min). The new controller reattaches through the existing durable-authority path; only the wait before that changes. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`, Anthropic), 1M context window, tool use and code execution, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 05:18:44 -06:00
DottaandClaude Opus 5.5	bd83715a6e	fix(recovery): bootstrap a remote follow-up again when a restart cut it off after retiring its finished owner (#20241 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A restart must resume in-progress runs on its own. Restart recovery classifies each claimed native run and the executor acts on that claim. > - A C1 follow-up on a remote sandbox first retires the finished owner of the previous turn (it archives the previous controller state), then launches a new runner. > - A restart in that window left the run with an inherited session checkpoint but no runner for this run. Recovery read the inherited checkpoint as proof of a live remote runner and claimed reattach. The executor then found no runner identity and failed with `durable_identity_unreadable`, so the user had to press Retry. > - This pull request makes recovery recognize that nothing reached a runner for this run, and makes the executor bootstrap again over the empty root. > - The benefit is that this restart window resumes the follow-up by itself. ## Linked Issues or Issue Description What happened? In restart-chaos Daytona K13 storm wave seed 9503, task t03 (codex, C1) graded R3 retry_needed. Follow-up run 0d8b7258 retired its previous owner at 10:07:32.988 ("Run 4ee1c19a finished without a saved runner checkpoint; starting a new runner"). A K1 kill at 10:07:34 cut it off before any runner start was requested. On restart, recovery claimed `reattach_remote_runner` and the executor failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The run ended with `native_continuation_requires_reconciliation` and a Retry offer. Expected behavior Recovery sees that no runner was ever requested for this run, claims `bootstrap_incomplete`, and the executor starts the new runner the first attempt was about to start. Steps to reproduce 1. Run a remote (Daytona) C1 follow-up whose previous turn finished. 2. Kill the server after the follow-up retires the finished owner and before it records `native.process_start_requested`. 3. Restart the server. See the `reattach_remote_runner` claim and the `durable_identity_unreadable` failure. Paperclip version or commit master at `3963d38560` ## What Changed - `native-restart-recovery.ts`: the `remote_runner_never_launched` proof also holds when the run carries only its continued session. That means a session checkpoint with a matching identity and no projected provider event, active turn, semantic result, terminal or pending runtime request, plus the predecessor's `sessionIdBefore`. The proof still requires no `native.process_start_requested`, no controller checkpoint for this run, no runner process identity and no `sessionIdAfter`. The classifier then sees no checkpoint for this run. - `native-session-executor.ts`: a `bootstrap_incomplete` claim over a root with no runner identity and no controller state quarantines that root and bootstraps again for a remote target too. Previously only a local target with no live runner process could do this. Such a root was never authenticated by a controller on this host. - Tests: four classifier cases (restart; a requested runner start, a pending runtime request and a projected provider event stay held). Two executor cases (created and replacement sandbox) restart the exact state the kill left after the retire. ## Verification - Red before the fix: the classifier test got a non-bootstrap claim. The executor tests failed with `runner_state_identity_mismatch: durable_identity_unreadable`, the error from the wave. Both pass after it. - `native-runner-restart-recovery.integration.test.ts` and `native-recovered-owner-followup.test.ts`: 239 passed. - `native-session-executor.test.ts`: 1,310 passed. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server`: passes. ## Risks - Low to medium. The proof widens only for a claimed remote run with none of its own start, checkpoint, runner or provider evidence. A pending runtime request in the inherited checkpoint still holds the run, as in #20226. - The quarantined root keeps the archived previous owner for inspection. The usual quarantine retention applies. - Previous-release story: no saved state or schema changes. A run that an older build left in this state is classified with the new rule at its next claim. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 05:17:15 -06:00
DottaandClaude Opus 5.5	780f378f60	fix(recovery): settle a local workspace command whose server process died mid-command (#20233 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A run prepares its execution workspace with local commands (`workspace_provision`, `worktree_prepare`, teardown, cleanup). The server records each command as a `workspace_operations` row. > - A restart can stop the server while such a command runs. The row then stays `running` forever, because no process or recovery path owns it. > - The restart-chaos harness found this. A K2 drain interrupted a run, then the server started `bash ./scripts/provision-worktree.sh` for the run and exited before it finished. The retry succeeded, but the old row stayed `running`. > - The ownership reconciler already settles finalize attempts whose process died. Local command operations had no owner stamp and no ownership kind. > - This pull request stamps each local command operation with the process that runs it and adds a `workspace_command_operation` ownership kind that fails an abandoned one. > - The benefit is that a restart never leaves a stuck workspace operation behind. ## Linked Issues or Issue Description What happened? In restart-chaos wave seed 9807 (local, R1, faults K1/K2), tasks t03 and t04 graded R6 with `running_workspace_operations`. A K2 drain interrupted the run with `server_shutdown_interrupted`. A `workspace_provision` operation started 0.3 s later in the same process and never finished. The retry run succeeded. The orphan operation stayed `running` forever. Expected behavior When the process that ran a local workspace command is gone, recovery closes the operation. It does not change the run or its outcome. Steps to reproduce 1. Start a run that provisions a worktree with a provision command. 2. Drain or restart the server while the command runs. 3. After the restart, see that the `workspace_operations` row for the command stays `running`. Paperclip version or commit master at `d2fe85e57c` ## What Changed - `workspace-operations.ts`: a local command phase (`worktree_prepare`, `workspace_config_freshness`, `workspace_provision`, `workspace_seed`, `workspace_runtime_provision`, `workspace_repair`, `workspace_teardown`, `worktree_cleanup`) without a runtime-control stamp now records `commandOwner` (the controller process stamp). This process tracks the operation as live while it runs. Finalize attempts and command operations share one live-id set. - `workspace-finalize-attempt-settlement.ts`: the finalize settle is now a factory. A second instance settles abandoned command operations with `abandonedCommand` provenance and an activity entry. Both instances skip a row with a runtime-control stamp. - `ownership-registry.ts`: new owner-gone-attestable kind `workspace_command_operation`. The owner is the `commandOwner` process. A row without a stamp is owned by its run. - `controller-instance-lease.ts`: a running command stamped with this boot holds the controller instance lease, like a finalize attempt. ## Verification - Red before the fix: the two new tests and the kind-list test failed. Green after it. - `server/src/__tests__/ownership-reconciliation-report.test.ts` (29 pass): a dead-process `workspace_provision` operation and an unstamped one with a terminal run are failed. A live, foreign-PID-space, active-run, unowned or runtime-control operation stays `running`. The recorder stamps `commandOwner` on a running provision command. - Related suites pass: `workspace-runtime`, `workspace-operations-reconciliation`, `workspace-runtime-control-recovery`, `execution-preparation-phases`, `controller-instance-lease`, `heartbeat-workspace-finalize-branch`, `execution-workspace-runtime-control-conflict` (265 tests). - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` in `server` passes. ## Risks - Low. The settle fails only a row whose stamped process is proven gone, or an unstamped row whose run is terminal. It compares the owner stamp in the same transaction, and it never touches an operation that this process still runs. - A failed command row is now visible as `failed` with an interruption message instead of `running`. Run outcomes do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 04:33:20 -06:00
DottaandClaude Opus 5.5	267f54ef72	fix(chat): make task-not-found, unreachable-MCP and board denials final for agents (#20208 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People talk to their agents in Agent Chat, and the agents act through the Paperclip API > - A board-only denial now tells the agent that it is final and where the person acts (#20151, #20182). Other refusals and errors still give no sign that they are final > - So the agent keeps trying until the reply deadline, and the person gets no answer. In chat evaluations this happened on a task lookup that returned 404, on an MCP server that Paperclip could not reach, on a model question, and after a final budget denial > - This pull request makes those answers say that they are final and what the person must do, and it tightens the chat guidance after a board-only denial > - The benefit is a prompt, correct reply instead of a long search or forbidden workarounds ## Linked Issues or Issue Description Refs #20151, #20182 (same remedy for board-only denials). What happened In four Agent Chat evaluation cases on a slower model, the agent did not reply before its deadline: - Asked to open a task from an old link, the agent got `404 {"error":"Issue not found"}`. It quoted the private-task guidance, then searched archived projects, activity and the old host for 3 more minutes. - Asked to connect an MCP server at an unreachable URL, the agent got `502 {"error":"fetch failed","code":"runtime_error"}` from the health check. It retried, then used `curl` on the host and gateway IP addresses to look for the server. - Asked which coding models an agent can use, the agent got the runtime's model list in its first minute. The catalog did not say how to list OpenRouter models or that these lists answer the question, so it enumerated OpenRouter with the shell until the deadline. - Asked to raise a budget, the agent got the final board-only denial. The chat guidance said "Complete other permitted work", so it probed approvals and other routes and replied too late. Expected behavior The answer the agent receives says whether it is final and what the person must do, with a link when the page is known. The agent replies at once. It does not search other routes, retry, or contact other network addresses. Steps to reproduce 1. Open Agent Chat with a CEO agent. 2. Ask it to open a deleted task by its old link, or to connect an MCP server at a closed local port. 3. The agent calls `GET /api/issues/{id}` (404) or `POST /api/tool-connections/{id}/health-check` (502 "fetch failed"). 4. The body says nothing about finality. The agent keeps searching or probing and replies late or not at all. ## What Changed - `GET /api/issues/:id`: for an agent, the 404 body adds `code: "issue_not_visible"` and a `remediation`. It says that the lookup is final, that the agent must not search other routes or open a similarly named task, and that it should reply now and offer sharing or pasting. A missing task, a private task and another company's task get the same body, so the body reveals nothing. People and other routes keep the plain body. - MCP health check and catalog refresh: when Paperclip cannot reach the server (connection refused, DNS, network, reset, timeout or TLS), an agent's error adds a `remediation`. It says the person must correct the URL or start the server, that the agent must not contact other hosts, ports or network addresses, and it links Connectors (`/<prefix>/apps`). Status, `error` and `code` do not change. - `AGENT_CHAT_DIRECTIVE`: after a board-only denial, the agent replies now instead of making more calls. "Complete other permitted work" is removed. - API catalog: `POST /api/companies/{companyId}/tools/connections` says it saves without a check and points MCP-by-URL setup at `tools/apps/connect`, which checks the server first and saves nothing for a fresh setup that cannot reach it. `GET /api/companies/{companyId}/adapters/{type}/models` says which lists answer a model question (the runtime's list with `provider`, `acpxAgent` for ACPX agents and `environmentId`, and `opencode_local` with `provider=openrouter` for OpenRouter keys), that the OpenRouter list is not a qualification, and that with those inputs these calls answer the question. `openrouter` is added to the documented `provider` values. - API catalog authorization: budget policy upsert, budget incident resolution, company and agent budget updates, and approval approve/reject are now listed as board-only. Their routes already call `assertBoard`, but the catalog said `board_or_agent`, so an agent explored them as its own action before it got the 403. - Tests: new `agent-refusal-finality.test.ts` (including the catalog actor for the budget and approval routes); the private-task access test now checks that a private and a missing task give an agent the same 404 body; the guidance test checks the new wording. ## Verification - `cd server && npx vitest run src/__tests__/agent-refusal-finality.test.ts src/__tests__/board-access-denial.test.ts src/__tests__/issue-access-grants-routes.test.ts src/__tests__/authz-existence-oracle-guard.test.ts`: all pass (embedded Postgres included). - `cd server && npx vitest run src/__tests__/error-handler.test.ts src/__tests__/generic-mcp-connection.test.ts src/__tests__/openapi-routes.test.ts src/__tests__/remote-http-rebinding.test.ts src/__tests__/cloud-portfolio-error.test.ts src/__tests__/database-backup-catalog-guidance.test.ts`: 183 tests pass. - `npx tsc --noEmit -p server`: passes. - Real chat journeys on the chat evaluation branch (runner-opencode): the results will follow in a comment. ## Risks - Low risk. HTTP status, `error` and `code` stay the same for each changed error. Agents get extra top-level fields (`code` on the task 404, `remediation`). A client that compares a whole agent 404 body for `GET /api/issues/:id` exactly will see a change. - The task 404 body is the same for missing, private and other-company tasks, so it adds no existence oracle. Other issue routes keep their plain body. - The guidance change can make an agent stop after a board-only denial in a request that had other, permitted parts. The agent still reports what it finished. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 04:30:19 -06:00
DottaandClaude Opus 5.5	d2fe85e57c	fix(secrets): delete a secret that a binding proposal referenced (#20193 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can propose a secret binding. A person approves or rejects it on a review card. > - Each binding proposal row refers to one source: a secret or a pending secret proposal. A check constraint enforces this. > - The secret reference used `ON DELETE SET NULL`. When a person deletes the secret, Postgres sets the reference to null. The check constraint then rejects the row. > - So a person cannot delete any secret that a binding proposal ever referenced. The API returns 500. > - This pull request deletes the proposal rows together with the secret, as the other source reference already does. Before the delete, it rejects pending proposals so their cards close with a reason. > - The benefit is that secret deletion works again and no review card points at a removed proposal. ## Linked Issues or Issue Description Refs #15644 (related: block secret deletion while bindings still exist). What happened `DELETE /api/secrets/:id` returned 500 for a secret that a binding proposal referenced. The server log showed `new row for relation "company_secret_proposals" violates check constraint "company_secret_proposals_shape_check"`. This happened for pending, rejected and approved proposals. It also happened after the person removed the agent's binding. Expected behavior The person can delete the secret. Proposals that referenced it go with it. A pending review card closes as rejected and tells why. Steps to reproduce 1. An agent proposes a binding for an existing company secret (`POST /api/agents/me/secret-proposals` with `secretId`). 2. Approve or reject the proposal, or leave it pending. 3. Delete the secret with `DELETE /api/secrets/:id`. 4. The response is 500 and the secret stays. ## What Changed - `packages/db`: `company_secret_proposals.secret_id` now uses `ON DELETE CASCADE`, the same as `secret_proposal_id`. Migration `0452_secret_proposal_secret_cascade.sql` replaces the foreign key (`DROP CONSTRAINT IF EXISTS`, then `ADD CONSTRAINT`). - `server/src/services/secrets.ts`: `remove` accepts an optional `beforeDelete` hook. The hook runs after the secret is marked deleted and before its row is removed. - `server/src/services/secret-proposals.ts`: new `rejectPendingBindingsForSecret`. It rejects pending binding proposals for a secret with a reason and returns them. This closes their review cards and writes the normal activity entries. `createBinding` now holds the secret row `FOR SHARE` in its insert transaction and refuses a deleted secret. So no new pending proposal can start after the delete marks the secret deleted. - `server/src/routes/secrets.ts`: the delete route rejects pending proposals in `beforeDelete`. For each one it sends the normal rejection notice and wake (`notifySecretProposalResolution`), so the proposing agent continues. - `server/src/__tests__/secret-proposals-routes.test.ts`: a regression test deletes a secret with a pending, rejected and approved binding proposal through the route. It checks the secret and proposal are gone, the card status is correct, and only the pending case sends one rejection notice and one wake. A second test checks that a binding proposal for a secret that the delete already marked deleted fails with 404. ## Verification - Red first: the new test failed on `origin/master` with the shape-check violation for all three cases. - `cd server && vitest run --config vitest.config.ts src/__tests__/secret-proposals-routes.test.ts`: 47 passed. - `vitest run src/__tests__/chat-run-milestone-governed-work.test.ts src/__tests__/agent-secrets-routes.test.ts src/__tests__/secrets-service.test.ts`: 112 passed. - `tsc --noEmit` for `server` and `packages/db`: pass. `pnpm --filter @paperclipai/db run check:migrations`: pass. - Found by the `secrets.secret-proposal` chat evaluation. Its cleanup deletes the fixture secret and failed on this error in runs 42563093640 and 42571127911. ## Risks - The migration only replaces one foreign key. It does not rewrite rows. Existing rows already have a non-null `secret_id` or `secret_proposal_id`, so nothing else changes. - Deleting a secret now also deletes its binding proposal rows. The activity log keeps the proposal history (`secret.proposal.*` and `secret.deleted`). Secret-kind proposal rows that created the secret keep `created_secret_id ON DELETE SET NULL`, so their history stays. - Other paths that delete secrets (company deletion, user secret cleanup) now cascade instead of failing on the same constraint. They do not reject pending cards first. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 03:54:07 -06:00
DottaandClaude Opus 5.5	df4562ab38	fix(native): an inherited checkpoint with a pending runtime request is evidence (#20226 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #20153 and #20214 let restart recovery set aside a remote follow-up's admission-rebound checkpoint when the checkpoint is blank and nothing of the run reached a runner. Recovery then restarts the bootstrap instead of blocking > - "Blank" checked the cursor, terminal turns, active turn, semantic result and terminal, but not pending runtime requests > - A rebound checkpoint that still carried an outstanding runtime request would therefore be treated as no evidence. The run would restart as a fresh bootstrap and drop that request > - This pull request also requires no pending runtime requests before the checkpoint counts as a blank admission checkpoint > - The benefit is that an outstanding request always keeps the run on its reconciling path ## Linked Issues or Issue Description Refs #20214 What happened? After #20214 merged, ponytail review noted that `admissionCheckpointOnly` (shared by #20153's `remoteRunnerNeverReceivedCommands` and #20214's `inheritedCheckpointOnly`) did not inspect `pendingRuntimeRequests`. A rebound checkpoint with a pending request would be demoted to "no evidence", and the run restarted as `remote_execution_never_claimed`, dropping the request. Expected behavior A checkpoint with any pending runtime request is evidence. The run is not restarted as a fresh bootstrap. Steps to reproduce 1. Claim restart recovery for an unclaimed remote follow-up whose rebound checkpoint carries a pending runtime request and no other state. 2. Before this change, recovery claims `bootstrap_incomplete`. Paperclip version or commit Current master after #20214. ## What Changed - `native-restart-recovery.ts`: `admissionCheckpointOnly` additionally requires `pendingRuntimeRequests` to be absent or empty. - `native-runner-restart-recovery.integration.test.ts`: a held case, an unclaimed remote follow-up whose inherited checkpoint carries a pending runtime request. ## Verification - Red/green: on master the held case is restarted as `bootstrap_incomplete`. With the change it is not, and the other cases are unchanged. - `vitest run` of `native-runner-restart-recovery.integration.test.ts` and `native-restart-recovery.test.ts`: 2 files, 201 tests passed (14 skipped). - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` (`server`): passed. ## Risks - Minimal. The change only narrows when a checkpoint counts as blank. Every run it affects keeps the existing, more conservative classification. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 03:31:35 -06:00
DottaandClaude Opus 5.5	8525bb534c	fix(chat): post a turn's answer when a card it did not create is pending (#20220 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People talk to agents in Agent Chat. The agent's final message becomes the chat reply. > - A chat thread can hold a pending card that the board addressed to another person, for example a human-only confirmation for Riley. > - When Alex asks the agent to act on that card, the agent correctly gets `403 interaction_human_only` and writes a final answer that explains the denial. > - The shared answered rule (`conversationTurnAnswered()`, from #20179 and #20204) counted any pending human-only card in the thread as the turn's own response. So the answer was suppressed, and the chat stayed `active` with no reply. > - This pull request limits that rule to cards that the turn's own run created. Other cards still hold the task status. > - The benefit is that the person always gets the turn's answer, unless the turn itself asked a question with a card. Live completion and restart recovery stay consistent. ## Linked Issues or Issue Description Refs #20179, #20204 (the shared answered rule this PR corrects). What happened In an Agent Chat, the board created a pending `request_confirmation` card with `resolverPolicy: human_only`, addressed to another user (Riley). The card has no source run. Alex wrote "I approve Riley's request." The agent tried to accept the card and got `403 interaction_human_only`. It then wrote a correct final answer: the card needs Riley. The run succeeded with a `yielded` result and a `response_wake` continuation. Run completion recorded `presentationDecision: none`. No comment was created. The conversation stayed `active`, so Alex never got a reply. Cause: `conversationTurnAnswered()` reused the status-gate query `pendingNativeInteraction()`. In conversation mode, that query excludes another run's card only when it is an ordinary question or an `anyone` confirmation. A human-only card addressed to someone else is kept. The rule then treated that card as the card that "presents the turn", although this run did not create it. Expected behavior A successful chat turn posts its final answer exactly once, and the conversation moves to `waiting`. Only a pending question or confirmation card that the turn's own run created presents the turn in place of its prose. A card from an earlier turn, or one the board addressed to another person, can hold the task status, but it does not suppress the answer. Steps to reproduce 1. Open an Agent Chat as Alex. 2. As the board, create a pending `request_confirmation` card in the chat with `resolverPolicy: human_only` and `addresseeUserId` set to another person. 3. Send "I approve Riley's request." 4. Let the agent try to accept the card, get `403 interaction_human_only`, and end its turn with a final answer. 5. See that no reply appears, `presentationDecision.commentAction` is `none`, and `conversationState` stays `active`. ## What Changed - `server/src/services/native-runtime/native-governance.ts`: `conversationTurnAnswered()` now checks only pending cards whose `sourceRunId` is the turn's run. `pendingNativeInteraction()` gets an optional `createdByRun` filter for this. The status gate (`pendingNativeGovernance()`) does not change, so another person's card still holds the task. - Both callers get the fix through the one shared rule `conversationTurnPresentsAnswer()`: live run completion (`run-completion.ts`) and background presentation recovery (`recoverCommittedNativeChatResponse`). - `server/src/services/heartbeat/run-completion.test.ts`: new live cases "someone else's confirmation card" (board-created, human-only, addressed to another user, no source run) and "earlier turn's question card" (created by an earlier run of the same agent). Each case now also checks that the conversation moves to `waiting`. - `server/src/services/native-runtime/agent-chat-response-recovery.test.ts`: new restart case for the board-created card, plus a `waiting` check after recovery. - `doc/execution-semantics.md`: state that only the turn's own card presents the turn. ## Verification - Thesis: the reply is suppressed because the answered rule counts any pending human-only card in the thread, not only this run's card. If this is true, a turn with only a board-created card addressed to someone else posts no comment on master, and restricting the rule to `sourceRunId = runId` posts exactly one comment and settles the conversation to `waiting`, with the own-card cases unchanged. - Failing first: without the fix, the two new live cases failed (`expected [] to deeply equal [answer]`). The new recovery case also failed (no comment). - After the fix: - `cd server && npx vitest run src/services/heartbeat/run-completion.test.ts -t "posts a waiting conversation turn"`: 9 passed (includes the two own-card cases, which still post nothing). - `cd server && npx vitest run src/services/native-runtime/agent-chat-response-recovery.test.ts`: 62 passed. - Both suites use a real embedded PostgreSQL database. - Turn start: no change. The rule runs only after the provider turn ends. ## Risks - Low. Only the presentation rule changes. The status gate, the conversation settle rule (it already required `sourceRunId = run.id`) and the external-chat path do not change. - Behavior shift: a turn now posts its answer while another run's or the board's card is pending. Before, the answer was lost. A card that the turn itself created still presents the turn, as before. - Previous-release state: no persisted shape change. A run that the previous release already recorded as `none` keeps that decision. Recovery does not re-present it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use (shell, file edits, test runs) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 03:22:01 -06:00
DottaandClaude Opus 5.5	fe72bc5f81	fix(native): roll back a warm rotation cut off before its new control plane once its runner is proven dead (#20218 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A warm follow-up rotates authority before it attaches. It archives the previous epoch (suspended runner state and settled controller) under `authority-epochs/`, then writes its own control plane > - #20146 taught recovery to roll back a rotation whose new control plane held only an undelivered `run.attach`, once its runner was proven dead > - A hard kill can land one step earlier, after the archive and before the new control plane exists. The root then holds only the archive, with no active control plane and no runner state. The executor read "no durable identity" and failed with `runner_state_identity_mismatch: durable_identity_unreadable` > - This pull request applies the same proven rollback to that earlier point of the rotation > - The benefit is that a hard kill anywhere in the rotation window resumes the follow-up from the previous turn ## Linked Issues or Issue Description Refs #20146 What happened? Local mixed restart-chaos wave w-20261109T081456-s9901 (master `09e24dd12c`), t15 (codex M1): - Run `4b85497f` started at 08:29:32 as a warm follow-up and requested its process, then K1 killed the server. - Recovery stopped the previous run's warm runner (pid 60249) and claimed `resume_dead_runner`. - The executor failed with `runner_state_identity_mismatch: durable_identity_unreadable`. The quarantined root held: - `authority-epochs/epoch-c2842347…` with the previous owner `dc7d447b`: runner `suspended`, every command completed; - no `control-plane/` state and no `runner/runner-state.json`, only provider state. A Retry ran fresh. Phase 3 also saw the same signature on Daytona; #20214 covers that remote shape. Expected behavior For a local runner proven dead (`resume_dead_runner`), a root that holds only the archive of a suspended previous owner of this session restores that epoch, and the follow-up continues from the previous turn. Without a dead-runner claim it still fails closed. Steps to reproduce 1. Start a warm follow-up and hard-kill the server after the rotation archives the previous epoch, before it writes the new control plane. 2. Restart. Before this change, the follow-up fails with `durable_identity_unreadable`. Paperclip version or commit `09e24dd12c` (wave). Branch rebased on current master. ## What Changed - `native-session-executor.ts`: - `undeliveredAttachRotationSource` becomes `interruptedRotationSource`, and the new `latestSuspendedArchivedEpoch` shares the epoch checks: named by its identity digest, a matching suspended runner state. - Besides #20146's undelivered-attach shape, it accepts a root with no `runner-state.json` and no active control plane (absent or empty). In that case the epoch must belong to this session (`normalizedSessionId`) and to a different run. - The rollback moves an active control plane aside only when one exists. - `native-recovered-owner-followup.test.ts`: #20146's rotation test adds the "cut off before its new control plane" shape. With a dead-runner claim it is admitted from the restored previous owner; without the claim it still fails closed. ## Verification - Red/green: on master, the new admitted case fails with exactly `durable_identity_unreadable`. With the change it is admitted. The fail-closed cases are unchanged. - `vitest run` of the recovered-owner, restart-recovery and bootstrap test files and `native-session-executor.test.ts`: 8 files, 1627 tests passed (14 skipped). - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` (`server`): passed. ## Risks - Low risk: - The rollback still requires a `resume_dead_runner` claim on a local runner, an absent runner state, and an archived epoch of this session whose runner is suspended and whose name matches its identity. - Restoring only moves files back within the root, and nothing is deleted. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 03:20:23 -06:00
DottaandClaude Opus 5.5	ad57d5cc12	fix(native): restart an unclaimed remote follow-up that carries only its inherited checkpoint (#20214 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A remote per-turn follow-up inherits the previous run's session checkpoint at admission, rebound to its own identity. Its session root still holds the previous owner's remote marker until the follow-up rotates authority > - If a restart lands before the follow-up's controller ever claims execution, recovery should restart its bootstrap: nothing of this run reached any runner > - Recovery's unclaimed-bootstrap check required no checkpoint and no provider evidence. The inherited checkpoint failed both, so recovery claimed `reattach_remote_runner` > - Even under a bootstrap claim, the executor skipped the per-turn projection restore for every restart claim. It therefore never rehydrated the previous owner's external marker, and failed with `runner_state_identity_mismatch: durable_identity_unreadable` > - This pull request does not count the inherited checkpoint as evidence for an unclaimed remote run, and lets a bootstrap claim restore the previous owner's projection, as a fresh follow-up start does > - The benefit is that a follow-up drained in its first second on Daytona restarts on its own ## Linked Issues or Issue Description Refs #20153 What happened? Daytona per-turn restart-chaos K13 storm w-20261109T074128-s9503 (revision `9046884811`), t26 (codex C1): follow-up run `f1ddc43d` started at 07:54:18 and was drained by SIGTERM at 07:54:19, during workspace staging. - It had not requested a runner process, and its coordinator was never claimed (provider attempt 0). - Recovery claimed `reattach_remote_runner` (`hasCheckpoint: true`, `hasProviderEvidence: true`, both from the inherited checkpoint). - The executor failed with `runner_state_identity_mismatch: durable_identity_unreadable`, because the session root held the previous run's external marker. A Retry succeeded. Expected behavior An unclaimed remote run whose only checkpoint is the admission-rebound one, with no durable controller record and no provider events, restarts its bootstrap (`remote_execution_never_claimed`). The executor rehydrates the previous owner's projection and starts the follow-up from it. Steps to reproduce 1. Start a remote per-turn follow-up whose session root holds the previous owner's external marker. Drain the control plane before the follow-up's controller claims execution. 2. Restart. Before this change, recovery claims `reattach_remote_runner` and the executor fails with `durable_identity_unreadable`. Paperclip version or commit `9046884811` (wave). Branch rebased on current master. ## What Changed - `native-restart-recovery.ts`: - #20153's admission-rebound checkpoint test moves earlier. A new `inheritedCheckpointOnly` holds for a remote sandbox run (not a plugin task) with only that checkpoint, no durable controller record, no staged attachment and no provider events. - The unclaimed-bootstrap path and its classification then treat that checkpoint as neither a checkpoint nor provider evidence. - `native-inherited-runner-retry.ts` / `native-restart-recovery.ts`: a warm continuation carries its settled predecessor's runner process identity until it launches its own runner, and t26's row still held pid 596 at the restart. - The new `carriesInheritedRunnerIdentity` reuses the same-run retry path's proof (the exact pid, group and start time a settled predecessor recorded). - The restart classifier then ignores that inherited identity on the unclaimed path, for a run that carries only its inherited checkpoint. - A process identity that is the run's own still holds the run. - `native-session-executor.ts`: `restorePerTurnRunnerdController` now also runs for a `bootstrap_incomplete` claim, but only when the root's reference identity belongs to a previous run. A claim on the run's own root returns early, as before. - Tests: - `native-runner-restart-recovery.integration.test.ts`: an unclaimed remote follow-up with only its inherited checkpoint restarts, with no process identity and with its predecessor's runner identity. It stays held with its own runner identity, or when a provider event was projected; - `native-recovered-owner-followup.test.ts`: a bootstrap claim over the previous owner's external marker is admitted, rehydrating the marker, keeping the backup and quarantining nothing. ## Verification - Red/green: both new tests fail on master. The classifier test claims `reattach_remote_runner`, and the executor test fails with exactly `durable_identity_unreadable`. Both pass with the change, and the held case holds. - `vitest run` over the restart-recovery, bootstrap, recovered-owner and failed-workspace-recovery test files and `native-session-executor.test.ts`: 9 files, 1715 tests passed (14 skipped). After the inherited-identity change, the restart-recovery, bootstrap, recovered-owner, failed-workspace-recovery and inherited-runner test files: 8 files, 407 tests passed (14 skipped). Without the identity check, the inherited case is not restarted. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` (`server`): passed. ## Risks - Low risk: - The inherited checkpoint stops counting only when the database shows no controller record and no provider event for the run, and the existing unclaimed-bootstrap conditions all still hold. - The executor change runs the existing fresh-follow-up projection restore, with all of its own checks, only for a root that still belongs to the previous run. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 03:10:23 -06:00

1 2 3 4 5 ...