PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-13 15:55:46 +01:00

Author	SHA1	Message	Date
DottaandClaude Opus 5.5	afcf34c9da	fix(evals): form repair-cost-recording's saved-usage debt and grade it by saved state (#20895 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat feature evals check that the chat agent helps a person with real product state > - The `budgets-costs.repair-cost-recording` case checks that the agent sends the person to the saved-cost retry control and then reports the true accounting result > - The grader checked the agent's replies with keyword regular expressions, and the scripted person gave the agent the harness's own accounting readback > - Keyword checks reject valid wording and accept wrong answers, so the case did not measure the promise > - The setup also never formed: the prerequisite run always settled, so no "Ready to record" saved usage existed > - This pull request forms that state with a temporary trigger on the case-owned database (no product change), and replaces the two keyword checks with checks of saved state, the agent's own product calls, and rendered links > - The benefit is a case that can run, and a grader that does not depend on wording and that still fails the relevant broken behavior ## Linked Issues or Issue Description What happened The case `budgets-costs.repair-cost-recording` graded the chat replies with keyword regular expressions (`honest-post-retry-accounting-result` and `guides-accounting-without-rerun`). The scripted follow-up message also told the agent the harness's accounting result. Separately, the setup never formed in paid runs: the prerequisite native run settles as an unpriced cost event (`missing_price`), not as a `retryable` saved-usage row. Accounting retries every 5 seconds, so a retryable row only exists while accounting keeps failing. Expected behavior The grader accepts any wording. It requires that the guidance before the retry links the Costs page, that the post-retry run reads saved cost data before it answers, and that the agent does not write cost, budget or accounting records or start work. Steps to reproduce 1. Read `gradeAccounting` in `tests/runner-e2e/chat-catalog-domains/identity-accounting.ts` on master. 2. Reword a valid reply (for example "Head to Costs and press the button there; the work stays as it is."). 3. See that the keyword checks reject it. Refs #20600 ## What Changed - New `tests/runner-e2e/chat-accounting-fault.ts`, following `chat-archive-storage-fault.ts`: a temporary trigger on the case-owned database refuses cost-event writes for the one prerequisite task. The product's own accounting transaction rolls back, so the finished run keeps its saved usage pending and the product's automatic retries keep failing. Refused writes are counted in a rollback-proof sequence. It is allowed only when this case is the single selected execution, and it never writes product rows. - Setup: the case creates the prerequisite task unassigned, arms the trigger, assigns the task, and waits until Costs shows the genuine `retryable` row. The trigger is closed on a setup failure and at cleanup. - Act: the trigger is dropped right before the person clicks "Retry cost recording". Automatic recovery can record it first; the saved accounting read afterwards classifies either result (an unpriced event still shows in data quality, so it counts as unresolved). - `chat-archive-storage-fault.ts`: export its owned-database resolver for reuse (no behavior change). - `identity-accounting.ts`: remove the reply keyword checks for the repair case. - Add `initial-guidance-links-saved-cost-retry`: a reply from before the retry has a rendered link to the Costs page (`/<prefix>/activity/costs` or the `/<prefix>/costs` redirect). - Add `post-retry-reply-reads-saved-accounting`: the post-retry chat run made a successful read of a company cost report, or of the cost summary of the prerequisite task (bound by its redacted input digest). - Add `no-agent-accounting-fabrication-or-rerun`: no successful chat-agent call writes cost, finance, budget or accounting records or starts run or task execution. A denied attempt is not a write. - `observe` collects the chat agent's redacted `call_api` uses and results (`leadApiCalls`). - The scripted follow-up now reports only the toast that the page showed. That toast says "Costs recorded." even when the usage still has no price. - Own `oracleVersion` (`paperclip.chat-catalog.repair-cost-recording.structural.v3`) and source files in code and in `chat-catalog-descriptors.json`. - Calibration tests: reworded valid replies pass; a missing or late Costs link, a run link only, a foreign link, no fresh read, a read in the wrong run, a denied read, an unrelated read, another task's cost summary, and agent cost or execution writes fail. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-accounting-fault.test.ts tests/runner-e2e/chat-archive-storage-fault.test.ts tests/runner-e2e/chat-catalog-domains/identity-accounting.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: all pass. The embedded-Postgres test drives the product's `accountRunCost`, `reconcileRunCosts`, data-quality report and board retry against the trigger: only the faulted task's run stays `retryable`, other debt in the company settles, and the retry records it once the trigger is dropped. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in the changed files. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <sha> --check`: unchanged. - Paid run still needed: `chat-features.runner-opencode.local.budgets-costs.repair-cost-recording`. ## Risks - Low risk. The change affects only this eval case. - The new read check needs the chat agent to have company cost-read access. Agents cannot read `costs/data-quality` (board only), so the agent uses the cost reports or the task cost summary. - The trigger runs DDL on the case-owned database only; the same isolation checks as the archive storage fault apply, and it is dropped before the retry and removed at cleanup. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 18:29:54 -06:00
DottaandClaude Opus 5.5	a75d7df9c7	fix(evals): gate steering approval-delivery and steer-active on real product state (#20899 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat feature catalog runs real native agents to prove the chat and task-steering promises. > - Two steering cases failed in setup before they could measure the product. > - `steering.approval-delivery` waited for a worker run to stay running while its question was pending. The product parks a native turn as soon as a durable question is saved, so that state never occurs. > - `steering.steer-active` waited for the queue to say steering is "available". The OpenCode harness reports that it cannot steer, so the queue says "unsupported". > - This pull request gates each case on the real product state and grades the behavior that state promises. > - The benefit is that both cases now measure their user promise on the evaluated profile, without faked runs or faked steering. ## Linked Issues or Issue Description What happened Both cases failed during setup on `runner-opencode`. `steering.approval-delivery` stopped with "The actual question fixture did not reach its running pending-input boundary": the worker's `request_human_input` call parked its turn (run succeeded, `reportedWorkDisposition: yielded`, continuation `response_wake` for the question). `steering.steer-active` timed out waiting for `steeringDisposition: "available"`; the OpenCode driver declares `steering: false`, so the queue reports `"unsupported"` and the page shows Interrupt instead of Steer. Expected behavior `steering.approval-delivery` should treat the saved yield and the pending question as the pending-input state, queue ordinary comments and the bound answer behind a real active worker turn, and prove one continuation receives the answer. `steering.steer-active` should grade same-turn steering when the harness can steer, and grade the honest fallback when it cannot: the message stays visibly queued, nothing is recorded as steered, and one normal turn consumes it. Steps to reproduce 1. Dispatch `chat-features.runner-opencode.local.steering.approval-delivery` on master. 2. See the setup stop after the worker run succeeds with a yielded result. 3. Dispatch `chat-features.runner-opencode.local.steering.steer-active` on master. 4. See setup time out while `GET /api/issues/:id/queued-comments` reports `steeringDisposition: "unsupported"`. ## What Changed - `runtime-approval-queue.ts`: gate the question on the parked turn (`pendingInputYield`: succeeded, yielded, `response_wake` keyed to the question). An ordinary human message then wakes a real second Morgan turn that waits at the task gate. Ordinary comments and the bound answer queue behind that turn. The run count goes from 3 to 4. - `runtime-approval-queue.ts`: remove the reply keyword checks (`acornNamingDirection` and the "I have answered" regex). Grade exactly one saved answer path instead: the human response control, or an attributed canonical response. - `runtime-active-steering.ts`: accept either definitive capability. Unsupported branch: no enabled Steer control, a visible Interrupt control, a steer attempt with the person's session refused with `409 steering_unsupported`, no `issue.queued_comment_steered` activity, the exact message still queued, and one continuation that consumes it and registers a rendered blue `website.html`. - `runtime-active-steering.ts`: remove the reply keyword checks and the `/blue/` match on the delivery document. The rendered page proves the color. - Update calibration tests in both directions, the workspace setup test, and the descriptors (`oracleVersion` bumps, `expectedRunCount` 4 for approval-delivery). The scenario YAML is unchanged. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains/runtime-active-steering.test.ts tests/runner-e2e/chat-catalog-domains/runtime-approval-queue.test.ts tests/runner-e2e/chat-catalog-domains/runtime-approval-workspace.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: 87 passed. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in the changed files. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - The live proof is a paid run of `chat-features.runner-opencode.local.steering.approval-delivery` and `chat-features.runner-opencode.local.steering.steer-active`. ## Risks - approval-delivery now depends on a human message waking the worker while its question is pending. The product does this (`supersedeOnUserComment: false` keeps the card open, and comments wake the assignee unless the task is done or cancelled). If the worker does not run the gate on that turn, setup fails with a named label. - steer-active on OpenCode now measures the honest unsupported fallback, not same-turn delivery. A steering-capable profile still runs the same-turn branch. - Low risk to the product: only eval code changes. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 18:28:44 -06:00
DottaandClaude Opus 5.5	ba5ba54439	fix(evals): name unmet completion-gate readiness fields and grade the gate read, not reply words (#20887 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog eval `tasks.completion-with-gate` checks that the CEO does not force Done while an independent human review is still required > - Its setup waits for a real Morgan implementation that awaits the configured review stage > - The setup timed out and did not say which state field was wrong > - The retained evidence shows a product defect: the native completion path never started the configured review stage (fixed in #20882) > - This pull request makes the readiness timeout name each unmet field, accepts valid product continuation runs, and replaces reply keyword grading with a structural check > - The benefit is that the eval measures the gate, and its setup failures are self-explaining ## Linked Issues or Issue Description What happened The eval failed with "Timed out waiting for Real prior Morgan implementation awaiting independent review". Morgan uploaded `website.html` and reported needs_review. The task became in_review through a generic review card, but `executionState` stayed null, so the poll (pending stage for Riley) never matched. The run finished at 229 s of the 240 s setup window. The grader also matched reply words with regular expressions. Expected behavior Setup reaches the configured review stage through supported product paths. A timeout names each unmet field. The grader proves the outcome from saved state and run receipts, not reply wording. Steps to reproduce 1. Dispatch `chat-features.runner-opencode.local.tasks.completion-with-gate` on master. 2. Read `server.log` and `snapshots/chat-final-run-ledger.json`: `authoritativeDecision: in_review`, `finalizationReasonCode: actionable_attention_pending`, no executionState. ## What Changed - `internal-task-completion.ts`: `unmetPriorDeliveryReview` lists each unmet readiness field. The poll uses it for `accept` and `timeoutDetail`. - Setup and grader accept one or more succeeded Morgan runs (a product continuation is valid). Any failed, cancelled or timed-out worker run is an unmet readiness condition, so setup stops before the paid CEO turn. The delivery must come from one of these runs. - The `truthful-outstanding-review` reply regex check is replaced by `checked-actual-completion-gate`: the CEO's chat run read the task (`GET /api/issues/{id}` or `get_task`, matched by record-bound receipt) and replied. The saved pending stage, unchanged policy and absence of a forced Done still prove the outcome. - Run bound 2 → 3, setup window 240 s → 330 s, attempt 540 s → 660 s. `oracleVersion` is `paperclip.chat-catalog.completion-with-gate.v2`. The descriptor is updated. - Cherry-picks both #20882 commits so the paid run exercises the product fix. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` on `internal.test.ts`, `internal-comment-reference.test.ts`, `internal-edit-ownership.test.ts`, `chat-catalog-contract.test.ts`, `chat-catalog-consistency.test.ts` (184 tests) and 12 other suites that reference the case (133 tests). - Calibration: two reworded valid replies pass. A missing gate read, an empty reply, a forced Done and a failed worker run fail. The retained product state (in_review, no executionState) produces named unmet fields. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - `npx tsc --noEmit -p tests/runner-e2e`: no new errors in the changed test files. The local worktree has unbuilt packages, so CI supplies the full typecheck. ## Risks - The reply's meaning is no longer machine-checked. A person must read the reply in the paid run. - The larger run bound and windows accept slower valid runs. Readiness still fails on any failed worker run. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 18:17:43 -06:00
DottaandClaude Opus 5.5	23909077ad	fix(retry): re-deliver the failed run's wake when the board retries it (#20970 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - When a run fails, the board shows a Retry button. The Retry starts a new run with the wake reason `task_retry`. > - A failed run can serve a wake that happens once, for example a scheduled task monitor check (`issue_monitor_due`). The monitor is consumed when it fires. > - The Retry run did not tell the agent which wake it retries. The agent saw only `task_retry` and the earlier interruption. > - In the restart-chaos harness, the agent followed its brief ("act only on a board comment or a scheduled monitor check"), decided that the Retry was not the monitor check, and changed nothing. The run succeeded, but the task stayed `in_progress` with no run, no wake and no monitor. The person clicked a Retry that worked mechanically but lost the work it was for. > - This pull request records the retried run's own wake reason on the Retry's continuation trigger (`trigger.retriedReason`) and shows it in the wake summary. > - The benefit is that a Retry re-delivers the same wake that failed. Dotta's rule: if a Retry button shows up, it still needs to work. ## Linked Issues or Issue Description What happened? Restart-chaos wave `w-20261111T230537-s6402` (Claude, Daytona warm, volume-free server mode, restart fault K1), task t03 (workload W2). The phase-2 monitor wake started run `60d821af`. That run failed with `native_session_interrupted`. The board Retry was accepted (HTTP 202). Retry run `a5a3958e` started 2 s later and succeeded. Its context had `wakeReason: task_retry` and `executionContinuation.trigger = { reason: "task_retry", sourceRunId: "60d821af…" }`, with no reference to `issue_monitor_due`. The agent's summary: "This wake was a task_retry resuming the previously interrupted run, not a new board comment or scheduled monitor check, so per instructions I changed nothing." The issue stayed `in_progress` with `executionRunId = null`, monitor status `triggered` and no `nextCheckAt`. The harness graded `R3:retry_did_not_complete` and U1 `no_owner_no_action 182s`. Expected behavior A Retry of a failed run re-delivers that run's wake. A Retry of a failed monitor check is handled as that monitor check. Steps to reproduce 1. Give an agent a task that acts only on a scheduled monitor check. 2. Let the monitor fire, and make the monitor run fail (restart fault K1 during the run). 3. Click Retry on the task. 4. Before this change, the Retry run's prompt says only `reason: task_retry`, and the agent does not do the monitor work. Additional context The same signature (`R3:retry_did_not_complete:W2C:K0`, Codex, monitor rewake run `bac90594` failed `native_session_interrupted`, Retry run `cfb24146` succeeded without the phase-2 marker) is in wave `w-20261111T192658-s5201` task t06. That case was attributed to the response-wake loop that #20935 fixed. Its shape matches this cause: the Retry run served the monitor wake but did not do the monitor work. That wave kept no database, so I could not read its run context directly. Refs #20935 ## What Changed - `buildExecutionContinuation` adds `trigger.retriedReason` when the current wake is a Retry (`task_retry` or `retry_failed_run`). The value is the source run's own wake reason. A Retry of a Retry inherits the first non-retry reason from the source run's saved continuation. - `ExecutionContinuationEnvelope.trigger` gets the optional `retriedReason` field. - `renderPaperclipWakePrompt` adds `- retried wake: <reason> (this Retry re-delivers the wake the failed run did not finish; handle it as that wake)` to the wake summary for fresh and resumed sessions. - New test in `server/src/services/execution-continuation.test.ts` on embedded Postgres: a failed `issue_monitor_due` run, a Retry of it, a Retry of that Retry, and a non-retry wake. ## Verification - Red on master: `cd server && npx vitest run src/services/execution-continuation.test.ts -t "carries the failed run"` failed with `trigger` = `{ reason: "task_retry", sourceRunId }` and no `retriedReason`. - Green: the same file passes (45 tests). - `tsc --noEmit` passes for `packages/shared`, `packages/adapter-utils` and `server`. - `pnpm verify` exits 2 (a shared type changed), so required CI supplies broad coverage. - Chaos: the existing cell W2 × K1 (Claude, Daytona warm) reproduces this when the fault lands in the monitor rewake run. W2C (Codex) shows the same shape. I did not rerun a wave for this PR. ## Risks - Low risk. The change adds one optional field and one prompt line. It changes no admission, lock, status or authority path. - The field is computed only for Retry wakes. Other wakes are unchanged. - Runs saved by an older version have no `retriedReason`. A Retry built from them reads the source run's `wakeReason`, which older runs already saved. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 18:04:19 -06:00
DottaandClaude Opus 5.5	ba905c7254	fix(evals): name each team member's readiness and grade configure-team structurally (#20879 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog eval `teams.configure-team` checks that chat gets an installed team ready member by member > - Its setup installs Product Engineering, removes the QA member's account, and requires a real probe split: QA missing an account, the senior coder ready > - The QA probe returned a generic `opencode_hello_probe_failed` error, so the setup stopped with "probes do not establish the required missing-versus-ready account split" > - The cause is a product classification gap: OpenRouter answers an empty key with HTTP 401 "Missing Authentication header", which the OpenCode probe did not treat as authentication (fixed in #20878, cherry-picked here) > - This pull request makes the setup blocker name each member's observed readiness, and replaces reply keyword checks with structural checks > - The benefit is that the eval can reach the chat journey and grades it without matching words ## Linked Issues or Issue Description Refs #20878 What happened `teams.configure-team` failed in setup. The QA member's probe was `fail` with `opencode_hello_probe_failed`. The senior coder passed. The setup error did not say which member was in which state. Expected behavior The QA member's probe reports the missing-authentication check, so the split forms and the chat journey runs. When the split does not form, the setup error names each member's observed status and checks. Steps to reproduce 1. Run `chat-features.runner-opencode.local.teams.configure-team` on master. 2. Read `snapshots/chat-catalog-team-initial-probes.json`. 3. QA shows `fail` with `opencode_hello_probe_failed:error`; the setup stops. ## What Changed - Cherry-pick of #20878: the OpenCode probe classifies a provider 401 as `opencode_hello_probe_auth_required`. - The setup blocker now lists each member as `name (expected missing account \| expected ready \| other member): status [non-info checks]`. - Grader: remove the member-name substring check and the "team is ready" regex. Identification is the rendered link to the missing member's own page, which the case already required. The followup is graded by a non-empty reply from its own succeeded run, not by the words "ready/configured/verified/account". Fresh per-member probes, preserved working configuration and revisions, and the attributed repair stay unchanged. - Calibration: reworded and non-English valid followups pass; missing handoff, empty followup, and failed followup run fail. A setup test with the recorded failure shape checks the new error text. - `oracleVersion` bumped to `paperclip.chat-catalog.team-configure.v3`; `parse.ts` added to `sourceFiles`. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains/external-team-configure.test.ts tests/runner-e2e/chat-catalog-domains/external-team-configure-prerequisite.test.ts tests/runner-e2e/chat-runtime-profile.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts` passes. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check` reports no change. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs` passes. - The paid run `chat-features.runner-opencode.local.teams.configure-team` is still to run. ## Risks - The removed "team is ready" regex failed open. A false whole-team claim in the reply now needs human reading of the transcript. The structural checks still reject a missing handoff, a missing repair, and any member that fails its fresh probe. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 17:26:01 -06:00
DottaandClaude Opus 5.5	b7446969d7	fix(evals): give choose-capable-target a saveable computer and grade it structurally (#20881 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that agents answer real user questions through the product. > - The case `execution-environments.choose-capable-target` asks "Which computer can run Morgan with Claude Code?" while environment reads are board-only. > - Its setup saved a built-in fake sandbox as the "unsupported" computer. The product refuses to save fake sandboxes (since #4449), so setup failed with 422 before any agent run, and the test API hid the reason. > - No computer that the product can save is advertised as unable to run Claude Code, so the original contrast cannot be built through supported APIs. > - This pull request changes the contrast to advertised support versus runtime health, grades the reply by structure instead of keywords, and makes setup failures show the product's validation reason. > - The benefit is that the case can measure its user promise, and a future setup 422 says why. ## Linked Issues or Issue Description What happened In the full chat catalog run, setup failed with `POST /api/companies/<id>/environments returned 422; body withheld`. A focused server test shows the reason: "Built-in fake sandbox environments are reserved for internal probes and cannot be saved." The fixture payload was `driver: "sandbox", config: { provider: "fake" }`. Expected behavior Setup builds its computers only through supported APIs, the agent run starts, and the grader checks the user promise without reply keywords. A setup validation failure states the product's reason. Steps to reproduce 1. Run `chat-features.runner-opencode.local.execution-environments.choose-capable-target` on master. 2. Setup posts a fake sandbox environment and gets 422. 3. `cd server && pnpm exec vitest run src/__tests__/environment-routes.test.ts -t "rejects persisted fake sandbox environments"` shows the same rule. ## What Changed - `runtime-capability-choice.ts`: setup saves an SSH computer on a reserved `.invalid` host (no key, never probed) beside the local fixture computer. Production capabilities advertise Claude Code on both. The `enableEnvironments` experimental setting is still set only through `PATCH /api/instance/settings/experimental` and restored in cleanup. - Grader: removed the reply regex and keyword checks. New checks: agent gets 403 on capabilities and probe; the reply renders a link to `/<prefix>/company/settings/instance/environments` (or a target's edit page); the reply has no board-only SSH settings (host, user, path); targets (including status and metadata) and the worker are unchanged; exactly one chat turn and no tasks. `verifyResult` clicks the rendered link and checks that the page shows both computers and loads the same capabilities. - `execution-environments.yaml`: context changed from "one target supports the harness and another does not" to support versus health, with the reason. Recorded in `chat-full-catalog-testops-manifest.json` `changedFields`, pinned in the manifest test, and the inventory digest is updated. - `oracleVersion` is now `paperclip.chat-catalog.runtime-capability-choice.v2` in code and in `chat-catalog-descriptors.json`; `sourceFiles` adds `internal-support.ts`, `server/src/routes/authz.ts` and `server/src/services/environment-config.ts`. - `api.ts`: cookie-session 400/409/422 failures now include the product's `error` message, at most 240 characters, with request string values (8 or more characters), the session cookie and long token-like strings redacted. `details` and all other statuses stay withheld. - Includes a cherry-pick of #20880 (agent environment denials link the environment settings page). Without it, the agent has no route to give. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` on `runtime-capability-choice.test.ts`, `runtime-computers.test.ts`, `chat-catalog-contract.test.ts`, `chat-catalog-consistency.test.ts`, `api-private-transport.test.ts`, `api.test.ts`, `api-response-reading.test.ts`: 73 passed. - Grader calibration: valid and reworded replies with the settings link pass. Missing, unprefixed, wrong-prefix, cross-origin and wrong-page links fail. Replies with the SSH host, user or path fail. Unsupported capability, reachable host, 200 boundary, changed target or worker, an extra run and a created task fail. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 passed. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in changed files. - Paid run still needed: `chat-features.runner-opencode.local.execution-environments.choose-capable-target`. ## Risks - Premise change: the case no longer has a saved computer that is advertised as unable to run Claude Code, because the product cannot save one. The "avoid an incompatible target" clause stays in the expected outcome, but no saved target exercises it. - The reply's claims about health are not graded by words. A reviewer must read the paid run's reply. - The 400/409/422 reason can show product text that the test API hid before. It is bounded and redacted. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 17:25:51 -06:00
DottaandClaude Opus 5.5	068d814735	test(chaos): run waves in Cloud's volume-free server mode; make W2 phase 1 yield (#20930 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness (`tests/restart-chaos/`) runs real agent tasks against an isolated control plane and injects faults > - Cloud servers run in the volume-free mode (`PAPERCLIP_EPHEMERAL_SERVER=true`, durable agent files, run logs in object storage) > - The harness never set that mode, so every wave ran a different server mode from staging. Paths that exist only in that mode were never tested. One of them is the per-agent directory operation lock, so `--fail-recovery-step directory-lock` could never reach its step with any workload > - The re-wake workloads W2 and W2C also lost coverage: an agent that reported phase 1 with disposition `done` closed the task, and the harness never sent the re-wake (four of eight Claude cells on 2026-11-11) > - This pull request adds `--ephemeral-server` and makes the W2/W2C brief ask for a `yielded` phase-1 report > - The benefit is that chaos waves can run in staging's server mode and the re-wake cells test the re-wake on every provider ## Linked Issues or Issue Description What is the problem? The restart chaos harness cannot run the control plane in the volume-free mode that Cloud and staging use. The `directory-lock` recovery step (`agent_directory.operation_lock.before_acquire`) is reached only for disposable remote copies, which need `PAPERCLIP_EPHEMERAL_SERVER=true`. W2 and W2C cells also stop before the re-wake when the agent reports phase 1 as `done`. What is the expected behavior? A wave can opt in to the volume-free mode on Daytona, and every restart of the wave keeps that mode. The W2 and W2C cells reach the re-wake. Steps to reproduce Run `--fail-recovery-step agent_directory.operation_lock.before_acquire:1` on Daytona with any workload: the check `recovery_step_failed_once` reports "never reached". Run W2 on Claude: the first run ends with the task `done` and `rewake_sent` is "never sent". ## What Changed - `assertIsolatedServerEnvironment` (`tests/runner-e2e/harness-env.ts`) takes `expected.generatedSecretsMasterKey`. It admits `PAPERCLIP_SECRETS_MASTER_KEY` only when that parameter is set and equals the value exactly. `PAPERCLIP_SECRETS_MASTER_KEY_FILE` stays refused. The check runs over the final server environment. - `--k15-fresh-home` implies `--ephemeral-server` and defaults `--storage` to `s3`. A fresh instance home without the volume-free mode loses the agents' instruction bundles, which staging keeps in Postgres and object storage. - `--ephemeral-server` (`config.ts`, `types.ts`, `main.ts`, `supervisor.ts`, new `ephemeral-server.ts` with its unit test `ephemeral-server.test.ts`): it needs `--storage s3` and `--env daytona` and refuses K14. Every server of the wave gets `PAPERCLIP_EPHEMERAL_SERVER`, `PAPERCLIP_DURABLE_AGENT_FILES`, `PAPERCLIP_SHARED_UI_ASSETS`, the wave's Postgres as `PAPERCLIP_DATABASE_LISTEN_URL`, one stable `PAPERCLIP_SECRETS_MASTER_KEY` (also written to the config key file) and `RUN_LOG_S3_*` in the wave's object store. - `rewake.ts`: the W2/W2C phase-1 brief asks for `paperclip_finish` with disposition `yielded` and a `response_wake` continuation. - W2/W2C tasks are created in `backlog` with their assignee, and a board comment starts phase 1 (`startComment` in `workloads.ts`, used by `observer.ts`). A passive board wait needs a comment-started run; without it the yield queues an immediate continuation and phase 1 wakes itself. The start comment is posted once (saved-comment check plus a stable `clientRequestId`), and the brief keeps phase 2 out of blocking `remainingWork`. - Volume-free mode drops the repository workloads (B1, B5, F1, R1) from the default list and refuses them when listed: their fixture is a `local_path` project source, which a volume-free server refuses. - `README.md`: a section for the new flag and a note on the brief. - `shape-cells.test.ts`: option parsing and refusals for the new flag. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/shape-cells.test.ts` passes (16 tests). - `node scripts/typecheck-test-projects.mjs tests/restart-chaos` passes. - Live proof wave (Daytona, Codex, W2, K1 and K15 with a fresh home, `--storage s3 --ephemeral-server`): wave `w-20261111T205259-s6301` (2 tasks). The first server and both replacements (K15 with a fresh home, then K1) started and reached ready in the volume-free mode. The server process had `PAPERCLIP_EPHEMERAL_SERVER=true`, `PAPERCLIP_DURABLE_AGENT_FILES=true` and `RUN_LOG_S3_BUCKET` set. Both W2 tasks left phase 1 `in_progress` (yielded), received the re-wake and ended `done`. The wave also found product faults (reported separately, not fixed here): a re-wake after K1 failed with `runner_harness_state_mismatch: backup_without_reusable_lease` and needed Retry. - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/support.test.ts` passes, including the new master-key admission test (a matching wave key passes; an inherited key, a mismatched key and `PAPERCLIP_SECRETS_MASTER_KEY_FILE` are refused). ## Risks - Low risk. The flag is opt-in and the harness is test-only. The brief change alters the W2/W2C prompt, so results from older W2/W2C seeds are not directly comparable. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 17:12:14 -06:00
DottaandClaude Opus 5.5	ef755cecc6	fix(evals): grade case facts by the saved fact, bind case kickoffs to the CEO run, stop block-on-design on a missing blocker (#20948 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog runs real chat journeys and grades saved product state > - Three cases failed in runs 43088288923 and 43101125481. In two of them the product did the right thing > - `cases.update-case-facts` used a word allow-list on a saved field. `cases.create-case` read a field the product never fills for a new task > - `task-coordination.block-on-design` timed out in the browser on a missing row, which hid the real cause > - This pull request grades what the user asked for, binds records by structure, and stops with the real cause > - The benefit is that each case fails only when the product fails, and says why ## Linked Issues or Issue Description What happened - `cases.update-case-facts`: the CEO saved `browser: "Mobile Safari only (bug reproduces only on mobile Safari)"` and kept the other fields. The grader allowed only the words "safari" and mobile qualifiers, so "bug reproduces" failed the first phase. The earlier rewording of the scripted follow-up was not the cause: it is sent only after this phase. - `cases.create-case`: the CEO created the case, the `workflow` and `customer_brief` documents and Morgan's kickoff in one run, before Morgan started. The grader required the kickoff's `sourceRunId` to be a CEO run. The product sets `sourceRunId` to the run that holds the task being handed off, so it is always null when the kickoff creates the task. The grade never passed, so the harness kept waiting. Morgan then found no app to test, handed the case back to the CEO, and the extra CEO run broke the run bound. - `task-coordination.block-on-design`: the CEO saved no Design blocker. The scripted human then waited 30 s for the "Blocked by" row and failed with a locator timeout. Expected behavior Each case accepts every valid product behavior and rejects the failures that matter. A failure names its cause. Steps to reproduce 1. Regrade the retained observation of run 43101125481 for `cases.create-case` with the old grader: `real-first-morgan-kickoff` and `required-documents-persist-before-investigation` fail although the records are correct. 2. Call `corrected({ browser: "Mobile Safari only (bug reproduces only on mobile Safari)" })` with the old grader: it returns false. 3. Run `task-coordination.block-on-design` when the CEO saves no blocker: it fails on a UI locator instead of on the missing blocker. ## What Changed - `cases.update-case-facts` (oracle v8): grade the saved correction structurally, with no word lists. The browser fact differs from the case's own setup value, and no field keeps the setup's stale literal "not yet confirmed", in the saved case and in the reviewer's handoff context. A person reads whether the wording says mobile Safari. - `cases.create-case` (oracle case-create.v2): bind the kickoff and its documents to the CEO chat run that was active when the kickoff was saved, not to `sourceRunId`. Admit one CEO run on the primary task woken by Morgan's own case handoff back to the CEO, and add a check that binds it to that handoff. Run bound is now 2 to 3. - Shared run policy (`chat-catalog-run-policy.ts`, `chat-catalog-contract.ts`): new optional `maximumLeadCaseHandbackRuns` for `source-settled-with-active-work`. The default is 0, so other cases grade exactly as before. No other oracle version changes. - `task-coordination.block-on-design` (oracle v9): after the first turn, stop with "The CEO did not save Design as a blocker on Website" when the edge is missing. - Descriptors updated for the three cases. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` on `integrator-case-create`, `integrator-case-facts`, `runtime-dependency`, `chat-catalog-run-policy`, `chat-catalog-readiness`, `chat-catalog-failure-retirement`, `chat-catalog-consistency` and `chat-ordinary-phase-callers` tests: all pass. - Calibration in both directions: - case facts: the retained values of runs 43088288923 and 43107042937 and four rewordings pass. The unchanged setup value, a kept stale literal (in any field), empty facts and a first turn that left the value unchanged fail. - create-case: the retained run 43101125481 regrades to pass. Documents from another run, a kickoff outside the CEO run, a kickoff by Morgan, a handback with no handoff, a handback woken another way and two handbacks fail. - run policy: an undeclared handback, a lead run woken another way and a second handback fail. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in changed files. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - End-to-end: chat eval run linked in a comment below. ## Risks - Low risk. Only eval code changes. The shared run-policy option is opt-in, and its default keeps the old result. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 16:59:53 -06:00
DottaandClaude Opus 5.5	c5e6ab5ca8	test(evals): retarget instance-settings from the removed Agent Chat toggle to the Cases experiment (#20897 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog eval `instance-operations.instance-settings` checks that a person who asks the chat agent to turn on an instance experiment gets the supported operator path, and that the setting really works > - The case asked "Enable Agent Chat on this instance." Agent Chat is always on since #19360, and the catalog says so (#20169), so the agent correctly answered "already on" and the scripted human could not find the removed toggle > - The premise is obsolete, so this pull request changes it explicitly to the Cases experiment (`enableCases`), which still exists and has a real route gate > - It also replaces a reply-wording regex with structural checks > - The benefit is that the case measures its original promise on today's product ## Linked Issues or Issue Description What happened In the full chat-catalog run, the case failed in the human journey: the locator `getByRole('switch', { name: 'Toggle agent chat experimental setting' })` found no element. The agent's reply ("Agent Chat is already enabled ... there's no toggle to change") was correct for the current product. The grader also used reply regexes (`experimental\|enableAgentChat`, `operator\|admin\|human\|settings`, and a negative "I have enabled" pattern). Expected behavior The case tests an instance experiment a person can still turn on. The agent cannot change instance settings, does not change the setting, and links Experimental settings. The person follows that rendered link, turns the feature on, sees it persist, and the feature's page opens. Grading reads saved state and rendered links, not wording. Steps to reproduce 1. Run `chat-features.runner-opencode.local.instance-operations.instance-settings` on master. 2. See the agent answer that Agent Chat is always on. 3. See the human step time out on the missing Agent Chat switch. Refs #19360, #20169. Product companion: #20894 (the denial and the catalog name the Experimental settings page). It is cherry-picked here so the paid run measures it. ## What Changed - Premise change (explicit): query "Enable Agent Chat on this instance." becomes "Turn on Cases for this instance."; context and expected outcome follow. Recorded in the manifest `changedFields`, pinned in the manifest test, inventory digest updated. - Setup: refuses a hidden Cases setting, starts Cases off (cleanup restores the target's value), and probes GET/PATCH experimental settings with the agent's own key (403 `board_access_required`, setting unchanged). - Human journey: checks that Cases is still off after the agent's turn and that `/<prefix>/cases` redirects to the dashboard. Then follows the Experimental settings link that the reply rendered (no typed-URL fallback), turns Cases on, checks it after reload, and opens the Cases page. - Grader: the `supported-human-path` regex check is replaced by `agent-links-experimental-settings` (rendered link to the page, and the person followed it). Saved-state checks cover the off route, the save, persistence and the open Cases page. - `oracleVersion` is now `paperclip.chat-catalog.privileged.instance-settings.v2`. `sourceFiles` follow the code. - Calibration: the valid reply and a reworded reply pass. Each missing state, a missing link, a wrong page, a foreign origin and a literal `<prefix>` link fail. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains/privileged-companions.test.ts tests/runner-e2e/chat-catalog-domains/privileged-budget.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: 25 passed. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 passed. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in changed files. - The paid end-to-end run is not done yet. ## Risks - The case now depends on the Cases experiment. If Cases becomes always on or is removed, the case needs another retarget. - The agent can still say something wrong in prose. The grader cannot detect a false "I turned it on" claim without wording checks, so a human reads the reply. The saved state proves that the agent did not change the setting. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 14:02:01 -06:00
DottaandClaude Opus 5.5	6dc36700b1	fix(costs): mark unpriced usage in the agent, model and project cost reports (#20889 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cost reports show what agents and projects spent > - A run can have usage with no known price; Paperclip saves it with costStatus "unpriced" and costCents 0 > - The summary and the provider, biller and user reports count those events, but the agent, agent-model and project reports did not > - An agent that read the by-agent-model report saw a native run at costCents 0 and told the person it cost $0.00 > - This pull request adds unpricedEventCount to those three reports and tells agents what it means > - The benefit is that missing charges are reported as unpriced, not as free ## Linked Issues or Issue Description What happened In the chat eval budgets-costs.spend-breakdown, the person asked "How much did Launch cost this week, broken down by agent?". The week included a real native run whose usage had no known price (data-quality reason missing_price). The agent read /costs/by-agent-model and /costs/by-project. Both returned costCents 0 for that usage and no unpriced count. The reply showed "1 native run ($0.00)" and gave the total as complete. Expected behavior The reports say which rows include unpriced usage. The agent says that usage is unpriced, marks the total incomplete, and links the record. Steps to reproduce 1. Record one priced and one unpriced metered cost event for an agent in a project. 2. Call GET /api/companies/{companyId}/costs/by-agent, /by-agent-model and /by-project. 3. See costCents counts only the priced event, and no field shows the unpriced event. ## What Changed - `costService.byAgent`, `byAgentModel` and `byProject` return `unpricedEventCount` (same filter as the summary: unpriced and not subscription_included). - Shared types `CostByAgent`, `CostByAgentModel` and `CostByProject` add `unpricedEventCount`. - The OpenAPI catalog entries for the cost reports that count unpriced usage now say that costCents counts priced usage only, that unpriced usage must be reported as unpriced (never $0.00), and that /costs/data-quality lists those runs. - Tests: embedded-Postgres service test (red on master, green here) and an OpenAPI description test. A UI test fixture adds the new field. ## Verification - `npx vitest run src/__tests__/costs-service.test.ts` (19 passed). The new test fails with the master service. - `npx vitest run src/__tests__/openapi-routes.test.ts src/services/native-runtime/runner-api.test.ts` (2146 passed). - `tsc --noEmit` for packages/shared and server. ## Risks - Low risk. One added aggregate column per report and an added response field. The UI does not read it yet. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 13:36:03 -06:00
DottaandClaude Opus 5.5	77757a3a93	fix(evals): serve case-facts its own evidence; accept block-on-design's blocked-interaction reply (#20891 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat-feature eval catalog runs real chat journeys and grades the saved product state > - Two cases failed in eval run gha-43069861174 because of the eval, not the product > - `cases.update-case-facts` gave its investigator the wrong tool content. `task-coordination.block-on-design` waited for a run count that the product does not promise > - A wrong fixture or a wrong wait hides real product results, so these must be correct > - This pull request gives the case-facts investigator the content its case would really serve, and lets block-on-design accept the product's bounded blocked-interaction reply > - The benefit is that both cases test their real feature again, and the checks still reject the failures that matter ## Linked Issues or Issue Description What happened `cases.update-case-facts`: the shared Website Brief app served the website prerequisite text ("record a short implementation note and complete the task") to the case investigator. The investigator said that the text did not match the case. It then asked the board for the missing evidence. The task went to `in_review`, so the harness's later board handoff got a correct 422 ("Resolve task mode, blockers, or review before handing off"). `task-coordination.block-on-design`: the CEO saved the Design blocker and also commented on the blocked Website task. The product woke Morgan in its bounded blocked-interaction mode (`dependencyBlockedInteraction: true`). Morgan only acknowledged the hold. The pre-approval wait required exactly one run, so it timed out. Expected behavior Each fixture gives the agent the facts its own scenario has. Each wait accepts every behavior that the product contract allows. Steps to reproduce 1. Run `chat-features.runner-opencode.local.cases.update-case-facts`. Morgan's continuation reads the website brief and asks the board for evidence. The review handoff gets a 422. 2. Run `chat-features.runner-opencode.local.task-coordination.block-on-design`. The CEO comments on Website. Morgan gets a blocked-interaction run. The first wait times out. ## What Changed - `startRunBriefGate` takes an optional `briefText`. The default text does not change for the other cases. - `cases.update-case-facts` serves "no further reproduction evidence has been received yet; the affected device is still unconfirmed" after release (oracle v7). - `task-coordination.block-on-design` (oracle v8): - The pre-approval wait accepts Morgan runs only when the server stamped them as blocked interactions on Website that name the Design blocker, and they succeeded. Any other Morgan run, a failed run, or a second CEO run rejects the wait at once. - New check: before approval, Website has no documents and its status is not `in_progress`, `in_review` or `done`. - After approval, exactly one Morgan work run must still produce the saved result. - Calibration tests: the valid interaction passes. An unstamped run, a wrong blocker, a wrong task, a failed run, a missing declared run, a document saved before approval, and a second Morgan work run each fail. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts --testTimeout=180000 tests/runner-e2e/chat-catalog-domains/runtime-dependency.test.ts tests/runner-e2e/chat-catalog-domains/integrator-case-facts.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: 34 passed. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in the changed files. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged. - A paid eval run of the three affected cases is in progress. The PR comments will have its result. ## Risks - Low risk. Only eval fixtures, waits and graders change. The other cases that use `startRunBriefGate` keep the same text. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), extended thinking, tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 13:32:27 -06:00
DottaandClaude Opus 5.5	5c440d8889	test(evals): make scripted chat input name its referent (accept-one-proposal, share-private-task) (#20896 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that agents in chat keep their user promises > - Two cases sent a scripted message whose referent was not clear in the conversation > - The agent then answered a different question, so the eval did not test its promise > - This pull request makes each scripted message name the thing it refers to, without telling the agent what to do > - It also replaces a reply keyword check with structural checks > - The benefit is that the two cases test their real promises again ## Linked Issues or Issue Description What happened `questions-and-approvals.accept-one-proposal`: the person accepted a "Create Acorn launch checklist" card. The CEO created the task once and replied with a new offer: move the task into todo and execute it. The eval then sent the same bare "Yes, go ahead with that." again. The CEO read it as a yes to the new offer and moved the task to todo. An assignment wake made a third run, over the declared bound of two. `tasks.share-private-task`: the eval sent "Share this private task with Riley." in the standing CEO chat. In that chat, "this" is the chat task itself. The CEO tried to share its own chat task and never linked the private task. The grader also matched reply words (Riley, owner/share phrases). Expected behavior A repeated acceptance must refer to the same proposal, so that the eval can show that one acceptance creates exactly one task. The share request must name the private task, so that the eval can show that the agent routes the owner to the task's sharing control and does not change its audience. Steps to reproduce 1. Run `chat-features.runner-opencode.local.questions-and-approvals.accept-one-proposal`. The CEO reply ends with a new offer, and the bare repeat accepts it. 2. Run `chat-features.runner-opencode.local.tasks.share-private-task`. The CEO acts on its chat task, not on the private task. ## What Changed - accept-one-proposal: the second human message names the accepted card by its visible title (`Yes, go ahead with the "Create Acorn launch checklist" request.`). The grader checks this exact scripted message. All other checks stay: one task, still in backlog, one decision bound to the first reply, two runs. Oracle version is now `paperclip.chat-catalog.accept-one-proposal.v2`. - share-private-task: the query is now `Share my private task "Private launch task" with Riley.` The change is recorded in the manifest `changedFields`, pinned in its test, and the inventory digest is updated. - share-private-task: the reply keyword check is removed. The grader requires a rendered link to the named private task. `verifyResult` has the owner open that link and open the task's Share control ("Who can access this task"). Then it confirms that Riley is still denied. The audience check also rejects a grant to any person who is not the owner. Oracle version is now `paperclip.chat-catalog.share-private-task.v2`. - Calibration tests: a bare repeat or a "move it to todo" repeat fails; a todo task fails; reworded handoffs pass; a link to the chat instead of the task fails; grants to Riley or to another person fail; public visibility fails. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` on the case tests, contract, consistency and every chat-catalog-domains test that imports the changed modules: 14 files, 285 tests pass. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: 10 pass. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: no change. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in the changed files. Other errors come from the local worktree dependency links. - Offline regrade of the retained share-private-task run: the run fails the handoff check (it linked only the chat). The same run with a link to the private task passes. - Paid eval runs for both cases are pending. ## Risks - Low risk. Only eval code, the scenario query and catalog metadata change. - The share-private-task grader no longer reads the reply. A false success claim with a correct link and no saved grant is not caught by code. A reviewer must read the reply in the paid run. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use, extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 13:12:09 -06:00
DottaandClaude Opus 5.5	c02de922af	fix(chat): deliver the person's time zone to chat agents (#20898 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - People talk to agents in chat, and agents create routines and other scheduled work from that chat > - Paperclip does not record the person's time zone, so a chat agent cannot read "every weekday at nine" or "today" in the person's zone > - The agent then saves schedules in UTC, and the person gets the work at the wrong hour > - This pull request saves the browser time zone with the person's personal preferences and gives it to chat agents in the task prompt > - The benefit is that schedules and dates from chat match the person's local time ## Linked Issues or Issue Description What happened A person in America/Chicago wrote in chat: "Every weekday at nine in the morning, summarize Launch for me." The agent saved the routine schedule `0 9 * * 1-5` with the UTC time zone. It replied that it used UTC because the person's time zone is not recorded. Expected behavior The agent knows the person's time zone. It saves one enabled `0 9 * * 1-5` schedule with the time zone America/Chicago. Its reply names the zone. Steps to reproduce 1. Open the board in a browser with the time zone America/Chicago. 2. Open a chat with an agent. 3. Send "Every weekday at nine in the morning, summarize Launch for me." 4. Look at the routine trigger that the agent creates. Its time zone is UTC. ## What Changed - Database: add the nullable `time_zone` column to `user_workspace_preferences` (migration `0468`). This is an instance-wide personal preference by user, as AGENTS.md section 5 permits. - Shared: `WorkspacePreference` has `timeZone`. `updateWorkspacePreferenceSchema` accepts an optional IANA `timeZone` (max 64 characters, checked with `Intl.DateTimeFormat`). `coordinatorWorkspaceEnabled` is now optional. The request must contain at least one field. - Server: the preference service returns `timeZone` and updates only the fields in the request. The activity log entry stays the same. The OpenAPI entry uses the shared schema. - UI: a new `useBrowserTimeZoneSync` hook in `Layout` saves the browser time zone one time when it is different from the saved zone. It is silent and best-effort. It uses the selected company as the audit context. - Prompt: for a conversation (any adapter), the task context reads the chat owner's saved zone. Recovery keeps the zone that the first turn admitted. The chat block adds one line only when the zone is known: `The person's time zone is America/Chicago (from their browser). Read times and dates they give, such as "nine in the morning" or "today", in that zone, and save schedules such as routine triggers with that timezone; say the zone when you report a schedule.` ## Verification - `cd server && npx vitest run src/__tests__/workspace-preferences.test.ts src/__tests__/workspace-preferences-routes.test.ts src/services/heartbeat/task-markdown.test.ts src/__tests__/heartbeat-task-context.test.ts src/__tests__/openapi-routes.test.ts src/services/native-runtime/runner-api.test.ts` - `cd ui && npx vitest run src/hooks/useBrowserTimeZoneSync.test.tsx src/hooks/useCoordinatorWorkspaceEnabled.test.tsx src/components/Layout.test.tsx` - Red/green: without the task-context and task-markdown changes, the two new prompt tests fail. With the changes, they pass. - `pnpm check:token-gates` is clean. `tsc --noEmit` passes for `packages/shared` and `packages/db`. - Manual: open the board in a browser with a non-UTC zone. Then ask a chat agent to create a weekday routine at nine in the morning. The trigger uses the browser zone. ## Risks - Low risk. The migration adds one nullable column. - The PUT body is now more flexible: `coordinatorWorkspaceEnabled` is optional. An empty body still gets a 400 response. - The prompt grows by one line only for conversations with a known zone. - The saved zone changes when the person uses a browser in a different zone. The latest browser wins. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 13:11:57 -06:00
DottaandClaude Opus 5.5	5a332517da	fix(evals): start the owned dev instance with a supported run command (#20885 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat eval catalog has a case that checks how chat handles a request to set up a throwaway Paperclip instance > - In this case, a scripted operator creates an owned development instance with the supported CLI after the agent answers > - The operator step always failed, so the eval never measured the agent follow-up > - The fixture started `paperclipai run` with an unsupported `--yes` flag, and it discarded the child output, so the cause was not visible > - This pull request starts the owned instance with a supported command, keeps the child output for diagnosis, and replaces reply keyword checks with structural checks > - The benefit is a valid eval that measures the case promise, and a readable cause if the operator step fails again ## Linked Issues or Issue Description What happened The `cli-operations.owned-dev-instance` chat eval failed every time with "Stopped waiting for Actual owned development target readiness: Owned dev server exited before readiness". The agent reply was correct. The scripted operator phase failed. The fixture ignored the child stdio, so the evidence did not contain the cause. Expected behavior The fixture starts the owned development instance with a supported CLI command. The instance becomes healthy. The eval then grades cleanup, isolation and the agent follow-up. If the child exits, the error names a bounded and redacted reason. Steps to reproduce 1. Start a disposable source instance with an embedded database. 2. Copy the checkout like the fixture does, then run `paperclipai worktree init --instance X --home H --from-config <source config> --server-port P --db-port D --seed-mode minimal`. 3. Run `paperclipai run --config <checkout>/.paperclip/config.json --yes`. It exits 1 with `error: unknown option '--yes'`. 4. Run the same command without `--yes`. It exits with `TS2688: Cannot find type definition file for 'node'` because the copied plugin SDK has no linked `node_modules`. ## What Changed - Remove `--yes` from the owned `paperclipai run` command. `run` has no such flag. It onboards only when the config is missing, and `worktree init` writes the config. - Link the installed `node_modules` of every workspace package into the copied checkout. Before, the fixture linked only six packages, so `run` could not compile the copied plugin SDK. - Write the owned CLI output to a private `0600` log under the attempt temp root, outside the owned root that cleanup deletes. - Add a bounded reason (last 12 lines, 1200 characters at most, terminal codes removed, known secrets and secret shapes redacted) to readiness and CLI failure errors. - Replace the `operator-dev-guidance` reply keyword check with `initial-chat-left-host-unchanged`. The check compares the digests of every Paperclip installation config under the attempt root (agent HOME, workspace and runtime directories) before and after the initial turn. - Change `native-reports-real-target` to require only the exact `host:port` that the human receipt supplied (bare, linked or with the scheme). It no longer matches outcome words. - Bump the case `oracleVersion` to `paperclip.chat-catalog.privileged.owned-dev-instance.v2` in code and in the descriptor. ## Verification - Local reproduction with a disposable source instance: the old command exits with `unknown option '--yes'`. Without `--yes` and with the old links, `run` exits with TS2688. With this change, the owned target reports healthy in 16 to 19 seconds. The seeded canary task reads back from the target. `worktree:cleanup` removes the instance. - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains/privileged-operators.test.ts tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: 17 passed. - Calibration: the retained CI reply and reworded receipt reports pass. A changed host installation, a missing target, a longer port, a different port, an unattributed reply and an empty initial reply fail. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD sha> --check`: no change. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in the changed files. Remaining errors come from a local dependency mismatch in unrelated packages. - The paid eval `chat-features.runner-opencode.local.cli-operations.owned-dev-instance` is not run yet. ## Risks - Low risk. The change is in one eval fixture and its unit test. - The installation inventory walks the attempt root. It skips `node_modules` and `.git`, and it stops with an error after 1,000,000 entries. - The initial-turn check does not grade the reply wording. A reply that falsely claims to have created an instance, without host changes, needs human review of the retained reply. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 13:04:22 -06:00
DottaandClaude Opus 5.5	683fd1e9a5	test(restart-chaos): cells for staging's no-restart failures (re-wake, Nth recovery step, long turn, K15 without volume) (#20893 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness (`tests/restart-chaos/`) tests that agent work survives control-plane faults > - In one 8-hour staging window, 20 of 25 failed runs happened with no restart at all, so the harness did not see them > - The harness had blind spots: one short task per agent, no wake right after a successful run, no fault in a later recovery step, short turns, and a redeploy that always keeps its instance home > - This pull request adds cells for those shapes, so the harness reproduces these failure classes on master > - The benefit is that fixes for these classes can be proven red and green with the harness, without staging ## Linked Issues or Issue Description What happened? On staging, an agent ran 10–13 tasks at once on reused sandboxes. 30–60 s after a long run succeeded, an ordinary wake (a comment or an issue monitor) started the next run of the same task. That run or its recovery failed before any provider work, with errors such as `runner_remote_process_identity_unavailable`, `Controller journal drain timed out` and `journal_tool_input_unanswered`. Other failures happened in a later recovery step (a held agent-directory lock, a missing runtime asset after a deploy without a volume). Expected behavior The chaos harness can reproduce each of these shapes on demand, and grades them: no failed attempt, every task `done`, and no task left blocked without a working Retry. Steps to reproduce See the commands in the README sections "Re-wake after success", "Fail the Nth recovery step", "Long turn" and "Redeploy without a volume, with overlap". Related: #20844 (session fault SR1). W2C is the configuration change after a successful run; SR1 is the same change while a question is parked. This PR does not add or change `session-faults.ts`. ## What Changed - New workloads W2, W2Q, W2C (`rewake.ts`): phase 1 ends with the task left open; the harness, as the board, wakes the same task the seeded 0–60 s after its run settled. W2 uses a comment or a board-scheduled issue monitor (by nonce). W2Q parks on an unanswered `request_human_input` question and is woken by a comment before the answer. W2C changes the agent's `adapterConfig.idleTimeoutMs` before the wake. Oracle: phase 1 succeeded, the wake was served by a run that succeeded, `no_failed_attempt`, `done`, markers. - `--faults K0`: cells with no fault, for workloads whose shape is the fault. - `--fail-recovery-step <step>[:<nth>]` (`recovery-step-faults.ts`): when the wave's first restart fault starts its replacement, the harness arms one crash point in a new fail mode; the nth hit throws the error of the real step, once. Steps: `runnerd-start` (`spawn paperclip-runnerd EAGAIN`), `teardown` (provider destroy did not finish), `directory-lock` (`Agent directory operation is already active`). Crossed tasks get `recovery_step_failed_once` and `no_failed_attempt`. - Crash points (`packages/shared/src/crash-points.ts`): fail mode in the control file (`<token> <name>:<nth> fail [pid=<n>]`), with an optional call-site error; `pid=` binds the line to one server process. The harness arms the step for the replacement only and disarms it once its receipt lands. New point `agent_directory.operation_lock.before_acquire`. The teardown point moved into its `try`, so fail mode behaves like a failed provider destroy; kill mode is unchanged. All points stay disarmed under `NODE_ENV=production`. The runner package's own crash-point copy is not changed (runner image content ID unchanged). - `--long-turn-minutes <n>` (`long-turn.ts`, 15–60, opt-in): every task's first turn runs rounds of real tool work for n minutes before its workload. Checks `long_turn_rounds` and `long_turn_run_observed`. - K15 `--k15-fresh-home` (Daytona) and `--k15-overlap-seconds <min>-<max>` (`redeploy.ts`, `supervisor.ts`, `edge-proxy.ts`): each deploy generation gets a new, empty instance home and no volume variables; the next generation boots beside the old one behind an edge proxy on the wave's port, takes traffic once ready, and the old one gets SIGTERM after the overlap. - `tests/runner-e2e/source-sdk-public-inputs.json`: refreshed pins for the changed shared crash-point files. - README: one section per new cell, with what it covers. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts` (48 files, 581 tests after rebasing on master, includes the new `shape-cells.test.ts`). - `pnpm exec vitest run packages/shared/src/crash-points.test.ts server/src/__tests__/crash-point-call-sites.test.ts` (fail mode, call sites). - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`. - Chaos waves on master `1d67e309e0` server code (Daytona image `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:99fc2671fc320816407f63cffd3acbd27471621febcd26494fe25bb5d7c5dd77`, content ID `1836c515…`), all RED unless noted: \| Cell \| Wave \| Result on master \| Staging class \| \| --- \| --- \| --- \| --- \| \| W2/W2Q/W2C, 10 tasks on one Codex agent, `--daytona-warm`, K0 \| `w-20261111T163206-s5201` \| 9/10 fail. 3 tasks (W2 t02, W2C t03, t06): first fault `native_runner_process_exited`, same-run reattach, `native.remote_runner_recovery_failed` with `stage: process_verification`, `reason: process_not_running`, `runnerLifecycle: ready`, `runnerSettled: false`, terminal `runner_remote_process_identity_unavailable`, "Workspace repair required". 3 tasks (t04, t08, t10): `native_restart_recovery_blocked` (`native_retry_source_superseded`) while its successor ran (R4 `concurrent_runs`). 3 W2Q tasks: the answer's continuation was cancelled `issue_not_in_progress` (status `blocked`), and the task needed Retry. 2 W2C tasks then looped: 60+ succeeded runs in 20 minutes on an `in_progress` task. \| D′/T terminal step reproduced with no restart, no governed wait and no PRP timeout (same error code and refusal fields). C+ retry outcome (ownership ambiguous) reproduced as `native_restart_recovery_blocked`. Not reproduced: `Controller journal drain timed out`, `did not durably suspend`, `journal_tool_input_unanswered`. \| \| Same, local `--lifecycle warm` \| `w-20261111T160103-s5101` \| 4/10 fail. 3/3 W2Q: continuation cancelled `issue_not_in_progress`, Retry needed. 1 W2C: the same `in_progress` run loop (39+ runs). \| The remote-runner classes need a remote runner; locally they do not show. \| \| `--fail-recovery-step runnerd-start:1`, K1 at 10 s, local A1 \| `w-20261111T162432-s5303` \| Fail. After K1 the first runnerd start of recovery failed (`spawn paperclip-runnerd EAGAIN`); the run ended `native_provider_terminal_failed` ("previous turn is no longer active", "Automatic recovery stopped") and needed Retry. Control, same seed and K1 without the step fault (`w-20261111T171114-s5303`): pass. \| A later recovery step that fails once ends the run instead of trying again (the A-chain shape). \| \| `runnerd-start:2`, local \| `w-20261111T170909-s5304` \| Step never reached: recovery started the runner once and succeeded. Grade fails `recovery_step_failed_once` (tested nothing). \| — \| \| `teardown:1`, K1, Daytona A1 \| `w-20261111T172048-s5312` \| Pass. The teardown failed once and the sweep settled it. \| Not reproduced. \| \| `directory-lock:1`, K2, Daytona A1 \| `w-20261111T171247-s5311` \| Step never reached by A1 after K2. \| Not reproduced; A1 recovery does not collect remote agent files. \| \| K15 `--k15-fresh-home --k15-overlap-seconds 30-150`, `--storage s3`, Codex and OpenCode, T1/C1 \| `w-20261111T173618-s5322` \| 1/4 fail (OpenCode T1): `native_continuation_requires_reconciliation` with "Agent directory belongs to a different execution environment"; the Retry runs failed `adapter_failed` "Configured instruction entry is missing"; the task rests `blocked` on a board recovery action, and two sandboxes were left (R6). One Codex T1 task finished before its fault (`fault_crossed_tool`). \| A-deploy: a controller-local file missing after a deploy without a volume. Here the missing file is the agent's instruction bundle, not a runtime-context asset; whether staging keeps instruction bundles outside the instance home is not verified. \| \| `--long-turn-minutes 15`, K1 at 600 s, local A1 \| `w-20261111T175505-s5401` \| Task pass (10/10 rounds, 992 s span, run 1019 s, K1 crossed mid-turn). The wave exited 2 only because I switched branches in the worktree during the wave. \| Shape check only. \| ## Risks - Product code changes are test instrumentation only: three `crashPoint` call sites take a fail-mode error, and one new call site was added. Disarmed (always in production) each call is one comparison. Moving the teardown point into its `try` does not change kill mode: the process dies at the same instant. - The new cells are opt-in. Default waves plan the same tasks as before for the same seed. - The W2 cells depend on model behaviour (phase 1 leaves the task open). The oracle records the task status at the wake. - `--k15-fresh-home` drops everything under the instance home. Whether staging keeps agent instruction bundles outside the instance home is not verified (see the K15 result). ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, agentic coding with tool use (shell, file edits, test and chaos runs) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 12:58:41 -06:00
Nicky LeachandPaperclip	5dfbb04b85	Poll asynchronous workspace provider preparation (#20865 ) ## Thinking Path > - Paperclip starts agent runs through a workspace provider when a project uses remote storage. > - The provider must prepare the workspace before the agent starts. > - One long worker call ties the checkout to its transport. > - Some checkouts take longer than the transport allows. > - This change lets an opted-in provider accept work and report its deadline. > - Core waits for completion through short status calls and stops at the provider deadline. ## Linked Issues or Issue Description What happened? A workspace provider can lose its reply when preparation takes longer than a single transport request. This stops run bootstrap before agent execution. Expected behavior Core should wait for a provider-confirmed workspace through short requests. The provider's deadline must bound the wait. Steps to reproduce Use a workspace provider that takes longer to prepare a checkout than its transport can remain open. Start a run that requires a new checkout. ## What Changed - Add an optional workspace preparation method to the plugin SDK. - Validate pending receipts and keep the original binding checks on completed replies. - Poll only during active preparation. Retry a lost status reply within the advertised deadline. - Preserve the original operation for providers that do not offer asynchronous preparation. - Reject deadline offers that exceed the existing lifecycle lock budget. Honor accepted offers without shortening them. - Checkout verification uses the existing synchronous RPC with a 10-second deadline and a 12-second transport timeout. [Core verification PR #20876](https://github.com/paperclipai/paperclip/pull/20876) ## Verification - All 63 workspace SDK and Core integration tests passed with temporary local PostgreSQL. The tests cover lost status replies, deadline expiry, oversized offers, and synchronous verification with an async-capable worker. - All 58 runner SDK input, evidence, and public type tests passed with the regenerated source pins. The local server typecheck passed. ## Risks - This change activates only when the provider advertises the method. Existing providers keep their current behavior. - Polling uses one-second intervals during active preparation. It stops at completion, failure, or the provider deadline. - No database migration is required. ## Model Used - OpenAI GPT-6 Codex with local code execution and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-11 10:33:02 -08:00
Nicky LeachandPaperclip	b2edea5754	fix(workspaces): verify existing worktrees before fresh retries (#20876 ) ## Thinking Path > - Paperclip manages work for AI agents. > - Retry checks whether the task can keep its existing workspace. > - A provider can complete checkout before Core receives the result. > - Recreating a completed worktree costs time and can change its base commit. > - Core uses the same ownership and writer checks for local and provider workspaces. > - The provider verifies its checkout before Core creates a fresh workspace. ## Linked Issues or Issue Description What happened? A retry can allocate a new provider worktree even though the original checkout completed. Steps to reproduce Complete workspace checkout through a storage provider. Fail Runner startup or lose the prepare reply. Retry the task. Expected behavior Reuse the same worktree when the provider confirms checkout and the prior run can no longer write to it. Related: #20865 addresses timeouts while the original prepare is in progress. This PR checks an earlier checkout when the user retries. ## What Changed - Check the prior task binding and writer state before asking a storage provider to verify a checkout. - Extend the existing prepare operation with an optional verification mode. Providers opt in through their manifest. - Return through normal workspace reuse after verification. Keep the fresh retry fallback when verification cannot establish checkout. ## Verification - Core workspace preparation, plugin workspace ownership, retry continuity, and SDK workspace-provider suites passed locally (213 tests). - Focused cases pass for provider verification, retry through heartbeat dispatch, and real local Git checkout verification with no plugin worker. The heartbeat test confirms that ordinary preparation and adapter dispatch keep the original workspace, branch, and base commit. - Core server typecheck passed before the final audit and timing edits. GitHub CI covers the final revision and broader regressions. ## Risks Provider verification adds one bounded call before a fresh retry. Older providers keep their current behavior. Verification must preserve the workspace, branch, and recorded base commit. The normal execution preparation checks still run before admission. No database migration is needed. ## Model Used OpenAI GPT-6 (Codex), with source inspection and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue details - [x] I have run targeted tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-11 18:04:18 +00:00
Nicky LeachandClaude Opus 5.5	369e238295	fix(projects): make workspace storage the project's only placement choice (#20835 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A repository project gets its working files from core staging or from a workspace storage plugin > - #20459 reported that an installed storage plugin attached every new repository project, so sandbox agents could not use those projects > - #20587 fixed creation with an Available in choice that selects storage > - #20651 then saved an environment on every new project and used it before the agent default to place runs. That solved a problem that did not exist: the project needs storage, not a run environment > - Creation saves the first environment of the chosen group, so a project silently sends every agent's runs to that environment. Project Configuration also shows two controls (Environment and Workspace storage) for one decision > - This pull request removes the project run environment and makes storage the only placement choice a project owns. Storage is fixed at creation > - The benefit is that runs follow the agent again, and project settings show one clear, read-only fact about storage ## Linked Issues or Issue Description Refs #20459. Follow-up to #20587 and #20651. Refs #11644. That issue asks for a project-level run environment override. This pull request removes the override that #20651 added for it, because the override is not necessary to fix #20459. #11644 stays a separate feature request. Related: #18176 (open) repairs placement through the legacy workspace-policy field. This pull request does not restore project placement, so it conflicts with the direction of #18176. ## What Changed - Remove the project run environment. Migration `0466` drops `projects.environment_id`. Run placement no longer reads the project. Runs use the agent default, then the instance default, then local execution. The low-trust task sandbox and the enforced sandbox policies keep their precedence. - Keep the #20651 recovery and binding guards. A recovered run still uses its admitted environment lease (source `run`). Company-binding checks before admission and during lease creation stay. - Keep the creation choice. Available in in New Project still selects storage. It still shows only when the environments need more than one storage owner. That can only occur when a storage plugin is installed. `POST /projects` with `environmentId` selects the storage of that environment and does not save the environment. - Fix storage at creation. `PATCH /projects/:id` rejects any storage change with `409 project_storage_locked`, from core to a plugin and from a plugin to core. Before, a core project could move to a plugin. - Drop legacy writes. Older clients can still send `executionWorkspacePolicy.environmentId`. The service removes it on create and update. - Simplify Project Configuration. Remove the Environment and Workspace storage selectors. Show an advisory Available in list of the environments that can use the project's storage. Show it when a storage plugin is installed or when the project uses one. When the project's plugin is not installed, show Storage plugin unavailable. Without a storage plugin, show nothing. - Issue workspace card. Remove the "project default" environment label. - Shared contract. Remove `Project.environmentId`, the `environmentId` project field, and `expectedStorageProviderPluginId` from the update schema. The create schema keeps `environmentId` and `expectedStorageProviderPluginId` from #20587. - Docs and maps. Update `doc/project-environment-compatibility.md`, `docs/api/goals-and-projects.md`, and the project feature and agent-access maps. - SDK source pins. Refresh `tests/runner-e2e/source-sdk-public-inputs.json` for the two shared files. The shared file blobs return to their values before #20651. The refresh also corrects the `pnpm-lock.yaml` pin and `lockSha256`: `master` pins an older lockfile blob than the one it contains. ## Verification - Targeted server suites pass (227 tests): `project-environments`, `project-workspace-provider.integration`, `heartbeat/environment-selection`, `execution-workspace-policy`, `environment-selection-route-guards`, `environment-run-orchestrator`, `openapi-routes`. - Related server suites pass (349 tests): `environment-service`, `environment-routes`, `plugin-workspaces.integration`, `issues-service`. - Targeted UI suites pass: `ProjectProperties.concurrency`, `IssueWorkspaceCard`, `NewProjectDialog`, `NewProjectDialog.managed-sandbox`, `ProjectDetail`. - New tests: - Creation saves storage, not an environment. Core and plugin projects both reject a storage change with `project_storage_locked`. - A legacy policy `environmentId` is dropped on create and update. - A new run follows the agent default for any project. A recovered run keeps its admitted lease after the agent default changes. - Project Configuration has no storage or environment selector. The note is hidden without a plugin. It lists the core or plugin environments for the project's storage. It shows Storage plugin unavailable for a removed plugin. - `pnpm --filter @paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui typecheck`, and `pnpm --filter @paperclipai/shared typecheck` pass. - `pnpm --filter @paperclipai/db check:migrations`, `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check` pass. - Not run locally: the full `pnpm test:run`, `pnpm build`, and browser suites. CI supplies that coverage. - Manual check: open Project Configuration on an instance with a storage plugin. Confirm that there is no Environment or Workspace storage selector, and that Available in lists the environments. ## Risks - Migration. `0466` drops `projects.environment_id` and its foreign key. The migration deletes the saved project environments. Apply it before you serve the new revision. An old server revision that reads `projects` fails until it stops. This is the same pattern as earlier drop-column migrations (`0229`, `0289`). - Behavior change for existing projects. Projects created after #20651 shipped have a saved environment that overrides the agent. After this change, their runs follow the agent default again. This is the intended fix. Before #20651, a project environment selection was not applied to runs (#11644), so no older behavior is lost. - No path from core to a plugin. A core project, including one created before a plugin was installed, cannot move to the plugin. To use plugin storage, create a new project. - API. A `PATCH` that sends `environmentId` or `expectedStorageProviderPluginId` no longer has an effect. The schema strips these fields. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with tool use (shell, file edits, test runs). Extended thinking was available. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 08:00:21 -08:00
DottaandClaude Opus 5.5	0b8ba1ac92	fix(queue): show messages a booting run's first turn takes as sent (#20822 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Task chat lets a person send follow-up messages to a running agent, and steer them into the live turn > - A sandboxed run (for example on Daytona) can take minutes to boot before its provider session exists > - A message sent during that boot cannot be steered, so it waits in the steering pane > - The runner then puts the message into its first turn, and the server removes it from the queue, but the view has no signal for this, so the message stays in the steering pane as "queued" > - That stale local entry also hides the queue id, so Steer and Discard fail for every later message > - This pull request tells the view which messages the first turn takes, and drops local queue entries whose run has settled > - The benefit is that a message sent while the sandbox boots shows as sent at once, and later messages keep working Steer and Discard ## Linked Issues or Issue Description What happened? A person sent a follow-up while the agent's sandbox was booting. The message stayed in the steering pane with Steer unavailable. After the agent started, the message stayed there. The agent had received it in its first turn. Later, Discard on that message failed with "Couldn't discard. Message is still queued.", and a new message also stayed in the steering pane. Steps to reproduce 1. Assign a task to a Paperclip Runner agent that runs in a Daytona sandbox. 2. While the sandbox boots, send a follow-up message. 3. Wait for the agent's first response. The message is still in the steering pane. 4. After the run hands off to a successor run, send another message and try to discard the first one. Expected behavior A message sent while the run is starting shows as a sent message. Later messages keep working Steer and Discard. Actual behavior The first message stays in the steering pane. Discard fails, and the new message cannot be steered until the page reloads. ## What Changed - `firstTurnQueuedCommentIds` (server, `first-turn-queued-input.ts`) returns the messages in a deferred queue that the target run's first turn already sealed, or will take while it is still starting. It uses the same eligibility rule as `claimFirstTurnQueuedInput`. It reads only lean columns, so the queue poll does not transfer run snapshots or large wake payloads. - The queue read path and the queue mutation path both put these ids in a new optional `firstTurnCommentIds` field on `IssueQueuedCommentQueue`. - The task view puts each of those messages in the conversation as input of that run, at its submission time, and not in the steering pane. - The task view no longer treats a saved message as a pending local queue entry after its target run is no longer live. A request that is still in flight keeps its original target. ## Verification - Red-green server test: `issue-queued-comments-routes.test.ts` "reports the messages a preparing/native run's first turn takes, before and after it seals them" and "reports no first-turn messages once the run's turn already started". Before the fix, `firstTurnCommentIds` was `undefined`. - Red-green UI tests: `IssueDetail.test.tsx` "shows a message the booting run's first turn takes as sent, not queued" (still starting, and sealed) and "keeps queue controls for a new message after the run that took an earlier one hands off". Before the fix, the queue held `['new-message', 'boot-message']` and the queue id was withheld. - Reproduced on staging first: the run's context held a first-turn receipt for the stuck message, and the server queue had no entries, while the view still showed it as queued. - `npx vitest run` for `issue-queued-comments-routes`, `queued-comment-consumption`, `queued-comment-postgres`, `queued-comment-use-cases`: 172 passed. The existing queue-poll byte budget tests still pass. - `npx vitest run src/pages/IssueDetail.test.tsx src/components/task-chat src/lib/issue-queued-comment-queue src/lib/optimistic-issue-comments`: 1288 passed. - `tsc --noEmit` for shared, server, and ui: no new errors. - `pnpm verify` exits 2 because the change touches `packages/shared`. Required CI supplies the broad coverage. ## Risks - Low risk. The new field is optional and only for display. Delivery still belongs to the first-turn claim and the deferred wake. - If a starting run fails before its first turn, the field goes away and the message shows in the queue again for the next run. This is correct, because the deferred wake still holds it. - The queue read adds one lean run read and one comment read only while a deferred queue targets a running run. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file edits, browser). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 15:52:14 +00:00
DottaandClaude Opus 5.5	4529dfb77f	fix(runner): keep Codex planning read-only on Daytona without network namespaces (#20820 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A planning task runs a native Codex agent in read-only planning mode before the user approves a Plan > - On a Daytona sandbox, every Codex planning run failed before its first turn: `error creating thread: failed to load AGENTS.md instructions for environment 'local': fs sandbox helper failed ... bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted` > - Codex 0.160.0 reads files through a filesystem helper when the permission profile restricts reads. That helper always runs bubblewrap with `--unshare-net`. Daytona has no `CAP_NET_ADMIN`, so bubblewrap cannot bring up loopback in the new network namespace > - The planning profile must stay read-only. Codex's plan collaboration mode is only a prompt, so the external-sandbox profile (#20808 tried it, then reverted it) would let planning runs write files > - This pull request keeps the read-only profile, but inside an external sandbox it gives the profile full-disk read, no writes and network. Codex then reads files directly and runs commands without a new network namespace, and it still refuses writes > - The benefit is that Codex planning works on Daytona and stays read-only ## Linked Issues or Issue Description Refs #20808 What happened? A Codex planning task on a Daytona execution environment failed at startup. The run ended with `provider_session_open_rejected` and the Codex message `bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. Standard and ask runs on the same sandbox worked. Expected behavior A planning run on Daytona starts, writes its Plan and asks for approval. Codex must still refuse file writes and mutating commands during planning. Steps to reproduce 1. Configure a native Codex agent with a Daytona execution environment. 2. Create a task in planning mode. 3. The first run fails before its first turn with the bubblewrap loopback error. ## What Changed - `createIsolatedCodexAppServerArgs`: when `PAPERCLIP_RUNNER_EXTERNAL_SANDBOX=1`, the `paperclip-runner-workspace-read-only` profile is now `{":root"="read"}` (plus the brokered `GH_CONFIG_DIR` write that planning already had). Its network still follows `PAPERCLIP_RUNNER_NETWORK_ACCESS`. It has no deny entries, so Codex 0.160.0 treats it as full-disk read and skips its filesystem helper, which always unshares the network. Local and SSH planning keep the narrower profile unchanged. - The Codex session context now records `rootAccess` `read` (planning) or `write` (default) in an external sandbox, instead of always `none`. - The Runner still sends `permissions: "paperclip-runner-workspace-read-only"` and the plan collaboration mode on `thread/start` and `turn/start` for planning. A new Rust test pins that for an externally sandboxed read-only launch (no `sandbox: danger-full-access`, no `externalSandbox` `sandboxPolicy`). - A new Rust test proves that a launch saved with the previous planning profile rules has the same stable launch identity as a launch with the new rules, so saved sessions carry forward. - New manual Product E2E suite `codex-planning-read-only` (`runner-codex` on local and Daytona). The planning turn runs one shell write, writes a Plan whose first line records the outcome, asks for approval, and finishes after approval. A grader reads the run events and fails if no write attempt was seen or if any write succeeded. - `docs/codex-driver.md` describes the external-sandbox planning profile, why it avoids the helper, and its read scope. ## Verification - Red on Daytona, master `123ee31552`: [run 43029207500](https://github.com/paperclipai/paperclip/actions/runs/43029207500), cell `core-compatibility.runner-codex.daytona.plan-revise-accept` failed twice with `error creating thread: failed to load AGENTS.md instructions for environment 'local': fs sandbox helper failed with status exit status: 1: bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted`. - Planning turn completes on Daytona: [run 43031462803](https://github.com/paperclipai/paperclip/actions/runs/43031462803), cell `core-compatibility.runner-codex.daytona.plan-revise-accept`. The planning run succeeded, saved a two-step Plan with the draft marker and opened a confirmation for that revision (all API checks passed). The cell then failed a later browser check: the Plan preview rendered as the `task-chat-plan-preview-fallback` card, which the harness does not accept. That is a separate UI placement gap. - Write attempt on Daytona: [run 43038547892](https://github.com/paperclipai/paperclip/actions/runs/43038547892), cell `codex-planning-read-only.runner-codex.daytona.plan-write-refused` on `f30aeb3e72`. The planning run succeeded and ran the shell write `printf probe > plan-write-probe-....txt \|\| echo WRITE_REFUSED`. It ended `failed` with exit 1 and no file, because bubblewrap could not start the command (`bwrap: setting up uid map: Permission denied`). The write was refused fail-closed by the sandbox launcher, not by a read-only filesystem error. The stricter grader in `a3d43b7bfd` reports this case as 'sandbox failed to start' rather than a pass. Shell commands in Daytona planning therefore do not run on the current image; the driver doc records this as a remaining gap. - Red/green unit test: `npx vitest run src/drivers/codex/codex-security-config.test.ts` (in `packages/paperclip-runner`). The new test failed on the unfixed code (the profile was `":root"="none"` with `CODEX_HOME` denied and network off) and passes with the fix. - `cargo test --locked -p paperclip-runner-core --test codex_provider -- external_sandbox` and `cargo test --locked -p paperclip-runner-core --lib -- external_sandbox_planning_profile_rules runtime_launch_rebinding`: pass. - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/codex-planning-read-only.test.ts tests/runner-e2e/catalog.test.ts`: pass. `pnpm test:e2e:runner:typecheck`: pass. - Real Codex 0.160.0 app-server probe (macOS) with the generated launch arguments and `PAPERCLIP_RUNNER_EXTERNAL_SANDBOX=1`: `thread/start` returned sandbox `{"type":"readOnly","networkAccess":true}` and loaded the workspace `AGENTS.md`; `command/exec` with the planning profile refused `printf x > plan-write-probe.txt` (exit 1, `Operation not permitted`, no file) and allowed `cat AGENTS.md`. Trace logs showed 0 filesystem-helper invocations with the new profile and 336 with the previous restricted profile. - Codex 0.160.0 source: `exec-server/src/fs_sandbox.rs` always sets `NetworkSandboxPolicy::Restricted` for the helper, `core/src/agents_md.rs` uses the helper only without full-disk read, and `linux-sandbox` adds `--unshare-net` only for a restricted or proxy-only network. ## Risks - Inside an external sandbox, planning commands can now read the whole sandbox filesystem, including the isolated Codex home. Default-mode turns in the same sandbox already have full read and write access, so this does not widen what the sandbox exposes. Local and SSH planning are unchanged. - Inside an external sandbox, planning shell commands fail closed on the current Daytona image because bubblewrap cannot map user IDs there. Planning still reads files and writes the Plan through Codex and Paperclip tools. - If a future Codex release adds deny-read entries or makes the helper mandatory, planning on Daytona could fail again. The new Daytona E2E suite detects that. - Upgrade safety: the profile filesystem and network keys are excluded from the stable launch identity, so saved sessions resume with the new rules (new Rust regression). No persisted schema, pack contract or ACPX profile changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, GitHub CLI) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 09:44:28 -06:00
github-actions[bot]andlockfile-bot	1d67e309e0	chore(lockfile): refresh pnpm lockfiles (#20845 ) ## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description What happened? Dependency manifests on master changed after the last committed lock refresh. Expected behavior Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. Steps to reproduce Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. - Resync the SDK public source proof's lock pins from the refreshed lock. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>	2026-11-11 15:28:49 +00:00
Nicky LeachandClaude Opus 5.5	ff8ec41e6a	feat(runner): upgrade ACPX to 0.22.0 and move patched behavior into Paperclip (#20770 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip Runner and the shared `adapter-utils` engine run ACP agents through the embedded ACPX library > - Paperclip pins ACPX 0.13.1 and keeps a large patch (`patches/acpx@0.13.1.patch`) on it > - ACPX 0.22.0 adds host hooks (`onAcpMessage`, `assertActive`, `processLifecycle`, prompt-response `_meta`, `spawnAgent`-friendly launch) that make much of that patch unnecessary > - A large patch is expensive to port and hides behavior that belongs to Paperclip > - This pull request upgrades to ACPX 0.22.0 and moves most patched behavior into Runner and `adapter-utils` > - The benefit is a patch that is 484 lines instead of 1,334, with the moved behavior covered by Paperclip tests ## Linked Issues or Issue Description Refs #20675, #14430, #15075, #17942, #9980 Proposed enhancement Upgrade the embedded ACPX library from 0.13.1 to 0.22.0. Keep only the patch behavior that stock 0.22 cannot express. Motivation The 0.13.1 patch held behavior that Paperclip owns, for example usage receipts, Copilot policy, model checks and the agent environment. Each ACPX upgrade had to port all of it. ACPX 0.22 gives hooks that let Paperclip do this work itself. Current behavior Runner and `adapter-utils` depend on 1,334 lines of patched ACPX 0.13.1 code. Expected behavior Runner and `adapter-utils` use ACPX 0.22.0. The patch keeps only: extension methods, Cursor subagent routing, event `_meta` and the message boundary chunk, the protocol guard hook, `spawnAgent`, and `requestExtension`. ## What Changed - ACPX 0.13.1 → 0.22.0 in Runner, `adapter-utils`, the server, the Daytona runner images and the qualified profiles (`acpxVersion`). `@agentclientprotocol/sdk` moves to 1.9.0. - New `patches/acpx@0.22.0.patch` (484 lines). The 0.12.0 and 0.13.1 patches are removed. - Delivery receipts: Runner confirms a permission reply when `onAcpMessage` sees the exact outbound reply. Question and extension replies count as delivered when Runner hands them back to ACPX. New `interaction-delivery.ts`. - Copilot policy: the existing policy function runs from `onAcpMessage`. A violation stops the agent, cancels later permission requests and fails the next prompt through `assertActive`. - Model selection: stock 0.22 validation, plus a sidecar gate (`createAcpxEffectiveModelGate`) that refuses a prompt if the agent reports another model. Grok binds its model when the session opens. Antigravity adds its concrete model to the model list ACPX reads. - Typed session failures: Runner and `adapter-utils` read them from the turn result `_meta` and fail the turn. They advertise the capability themselves. - Pi and Cursor usage receipts: a collector (`prompt-receipts.ts`) binds them to the turn's own `session/prompt` reply. - Goal capability: Runner reads it from the `initialize` reply. It keeps it under `paperclip.goalCapability` in the saved agent capabilities `_meta` and restores it when a saved session loads. - Agent environment: Runner and `adapter-utils` set the agent environment in `spawnAgent`. ACP terminals keep their current environment. `adapter-utils` blocks inheritance of `XAI_API_KEY` and refuses to start if the server has an `ACPX_AUTH_` variable. - `adapter-utils` spawn hooks (`spawnCwd`, stderr, `onAgentSpawn`): a new `stockAcpRuntimeOptions` maps them to stock `spawnAgent` and `processLifecycle`. - Windows: ACPX 0.22 rejects raw command strings on Windows. `adapter-utils` now passes an argv array there. - Grok rules: a Runner guard (`grok-session-rules.ts`) writes `session/new` `_meta.rules`. - Turn-dispatch journal (#20675): ACPX 0.22 ignores errors from `onAcpMessage`. The adapter keeps a failed `dispatched` write and throws it from the turn's `assertActive`. ACPX calls `assertActive` before the prompt write, so the write still stops. The real-ACPX test now checks this route. `docs/durable-recovery.md` is updated. - Plan events: the sidecar reads 0.22 `status` events tagged `plan`. - CI: the backup-checkpoint lane now installs through the shared `setup-ci-dependencies` action, like PR CI, so it resolves a stale `pnpm-lock.yaml` in place. It runs on any root `package.json` change, and pull requests never commit the lockfile, so its own frozen install failed on dependency changes. - Tests that checked removed patch internals are removed or rewritten: the response-delivery, Cursor-usage and Pi-receipt package contracts, and the goal-persistence packaging test. New unit tests cover the moved behavior. ## Verification Local runs on macOS (darwin-arm64), Node 26, on the rebased head: - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/drivers/acpx src/cli`: 2,000 passed. - `tsc -p packages/paperclip-runner/tsconfig.json` and `tsconfig.test.json`: no errors, except the existing `@paperclipai/paperclip-eval-kernel` build gap in `workflow-harness.ts`. - Runner package contracts (`node --test test/acpx--package-contract.test.mjs test/acpx-sidecar-contract.test.mjs`), `scripts/build-inputs.test.mjs`, `scripts/contracts/contract-fingerprints.test.mjs`, `scripts/acpx-patch-packaging.test.mjs`: all pass. `node scripts/generate-acpx-sidecar-contract.mjs --check` passes. - `cargo test -p paperclip-runner-core --test acpx_park_preparation --test acpx_park_readiness --test acpx_warm_attachment --test native_provider_backend`: 86 passed. `cargo fmt --check` passes. - `adapter-utils`: `tsc` clean. `vitest run src/acpx-engine src/mcp-isolation.integration.test.ts`: 457 passed (one Gemini flag test failed once under load and passes when run alone). - Adapters: `claude-local` and `codex-local` `acp.quota.test.ts` (23 and 39 passed), `grok-local` (149 passed). - Server: `scripts/prepare-runner-vendor.test.mjs`, `native-session-executor.test.ts`, `quarantined-task-retry.test.ts` (1,416 passed). - `cargo test -p paperclip-runner-core --lib -- previous_acpx_release acpx_upgrade unrestorable_saved_sessions`: 3 passed. `node --test test/acpx-session-carry-forward-package-contract.test.mjs`: passes. - `patch -p1 --forward` on a clean `acpx-0.22.0.tgz` gives output identical to the port. - Under full-suite load, some Runner live and recovery tests time out. Each one passes when run alone. Full regression is left to CI. Not run: live provider runs (they need provider credentials), the opt-in Kimi live peer test, the Gemini filesystem smoke test, and Windows. ## Risks - Previous-release story: sessions saved by the previous release (ACPX 0.13.1) carry forward. ACPX 0.22 loads the session records 0.13.1 wrote. Runner moves a saved descriptor that names 0.13.1 (`CARRIED_ACPX_RELEASES`) to 0.22.0, and the sidecar keeps 0.13.1 as the ACPX value in the session identity, so the saved session key and identity record still match. The existing launch-profile rebind still decides which sessions resume: a settled session, or the same run's interrupted turn through lost-turn recovery. A session saved by any other ACPX release is still refused as `provider_state_incompatible`. `rules/acpx-profile-identity.mdc` and `docs/durable-recovery.md` now describe this. - Proof: `acpx-session-carry-forward-package-contract.test.mjs` resumes a session record written by the real ACPX 0.13.1 with its Paperclip patch. 0.22 loads the same provider session (no `session/new`), runs a turn and keeps the earlier turn. Rust tests cover a settled 0.13.1 session that the next run attaches to, and a turn interrupted by the upgrade. A recovery-identity test checks that the session key and identity record match across the versions. - `pnpm-lock.yaml` is not committed. CI resolves it for each run, and the lockfile refresh workflow updates it after merge. - Behavior change on the server: `adapter-utils` refuses to create an ACPX runtime if the server environment has an `ACPX_AUTH_*` variable. Before, ACPX ignored such variables. Stock 0.22 would use them to authenticate agents with server credentials. - Delivery confirmation is now Runner-owned. A permission reply is confirmed by the exact outbound message. Question and extension replies are confirmed when Runner hands them to ACPX, not when the pipe accepts them. - Users can select only models that the agent advertises. Stock 0.22 rejects other model IDs (Claude is exempt). - ACP terminals keep the sidecar or server environment, as before. This existing exposure is not changed by this pull request. - The `adapter-utils` Gemini version probe now runs with the server environment plus the run variables, not the run variables alone. - The pack contract digest and profile `acpxVersion` change, so Daytona needs a new runner image. - Grok launch flags (`--model`, `--reasoning-effort`) were checked on 1.0.13 but session open needs a login, so Grok keeps the existing `_meta` path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file editing, subagents). Extended reasoning was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 07:28:00 -08:00
DottaandClaude Opus 5.5	34f2bb78ee	feat(connections): connect Sentry with a long-lived user auth token (#20823 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents use app connections to call provider tools, such as the Sentry MCP server > - The Sentry connection has only one method: browser sign-in through the hosted MCP OAuth server > - That grant expires. An unattended triage agent loses Sentry access until a person signs in again > - During that time, a scheduled triage routine completes each run but reads no Sentry data > - This pull request adds a second Sentry method that uses a user auth token > - The benefit is a Sentry connection that stays valid until a person revokes the token ## Linked Issues or Issue Description Subsystem affected App connections: the Sentry entry in the connection catalog. Problem or motivation A Sentry connection that was created with browser sign-in on 2026-10-11 changed to "OAuth authorization expired. Reconnect this app to continue." at 04:01 UTC on 2026-11-10. The agent that used it had no Sentry tools after that time. Each hourly triage run reported the blocker but could not continue. Unattended agents need a Sentry connection that does not need a person to sign in again each month. Proposed solution The hosted Sentry MCP server accepts a Sentry user auth token in the `Authorization: Sentry-Bearer <token>` header. It reserves `Bearer` for its own OAuth tokens. Paperclip API-key methods already support a custom header prefix. Add a Sentry API-key method that uses this header. Alternatives considered Reconnect with browser sign-in again. That restores access only until the next grant expiry. ## What Changed - Add the `mcp-user-token` method ("Use a user auth token") to Sentry in `scripts/ingest-app-definitions.mjs`. It uses the same server URL and sends the token as `Authorization: Sentry-Bearer <token>`. - Regenerate `packages/shared/src/app-definitions/sentry.json`. I did not include unrelated output that the content corpus drift caused for other providers. - Add a permission review entry for the new method in `doc/connections/tool-method-permission-reviews.json`. - Add a test for the new method and its header prefix. - Document the token method, its required scopes, and its ownership risk in `doc/connections/SENTRY.md`. ## Verification - `pnpm exec vitest run packages/shared/src/app-definitions.test.ts`: 40 passed. The new assertion fails on `master` because Sentry has only one method there. - `pnpm exec vitest run src/__tests__/tool-access-service.test.ts -t Sentry` (server): 2 passed. - `pnpm exec vitest run ui/src/pages/apps/app-connect-policy.test.ts server/src/__tests__/satisfied-connection-intents.test.ts`: 64 passed. - `pnpm verify` returned exit 2 (a doc is a global CI input). Required CI supplies the broad coverage. - Live probe of `https://mcp.sentry.dev/mcp` with an `initialize` request: `Authorization: Sentry-Bearer <invalid token>` starts an MCP session (the token is checked on tool calls). `Authorization: Bearer <same token>` returns `401 invalid_token`. This confirms the server accepts the header scheme. - Not done yet: a real agent tool call with a real token after deployment. ## Risks - Low risk. This adds a method. The existing OAuth method and saved connections do not change. - A user auth token acts as its creator and lasts until revoked. The method guidance and docs say this. Rotate the token when that person leaves. - If Sentry changes the `Sentry-Bearer` scheme, connections that use this method fail on tool calls. The OAuth method still works. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, browser, web search). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 09:26:24 -06:00
Nicky LeachandClaude Opus 5.5	6ce953ca52	fix(runner-e2e): resync the SDK lockfile pins to master's current lock (#20834 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner support suite proves the plugin SDK public types from pins in `tests/runner-e2e/source-sdk-public-inputs.json` > - Two of those pins follow the root lockfile: the `pnpm-lock.yaml` blob, and `lockSha256`, which must equal the `outputSha256` in `tests/runner-e2e/report-lock.json` > - The lockfile refresh #20824 changed the lockfile and `report-lock.json`. The registry kept the old values > - #20623 makes the refresh resync these pins, but its registry values come from the refresh before #20824 > - The refresh resyncs only when it commits a lockfile change. The lockfile is current, so the refresh after #20623 did nothing, and the pins stay stale > - This pull request regenerates the two pins with the existing `--write` command > - The benefit is that `ci / Runner support suite` passes again on master and on new pull requests ## Linked Issues or Issue Description Refs #20824. Refs #20623. What happened? `ci / Runner support suite` fails on master at `e2cf9a670f` and on pull requests that merge master. `source-sdk-public-type.test.ts` and `source-sdk-evidence.test.ts` fail with "Reviewed SDK reference policy does not match public proof". `source-sdk-public-inputs.test.mjs` reports `pnpm-lock.yaml (changed)`. The refresh run after #20623 merged (run 43043576648) logged "Lockfiles and reporting-lock proof unchanged, nothing to do". Expected behavior The registry pins the committed lockfile, and `lockSha256` equals the `outputSha256` in `report-lock.json`. Steps to reproduce 1. Check out master at `e2cf9a670f`. 2. Run `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/source-sdk-public-type.test.ts tests/runner-e2e/source-sdk-evidence.test.ts tests/runner-e2e/source-sdk-public-inputs.test.mjs`. 3. See the "Reviewed SDK reference policy does not match public proof" failure. ## What Changed - `tests/runner-e2e/source-sdk-public-inputs.json`: regenerated with `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --write`. Only two values change: - `inputs["pnpm-lock.yaml"]`: `6d825f97…` → `d24edac1…`, the committed lockfile blob. - `lockSha256`: `b3208306…` → `17625672…`, the `outputSha256` in `report-lock.json`. - No other pin changes. The lockfile and `report-lock.json` do not change. ## Verification - Before the change, the three SDK proof test files fail with the policy mismatch. After the change, they pass. - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/source-sdk-public-type.test.ts tests/runner-e2e/source-sdk-evidence.test.ts tests/runner-e2e/source-sdk-public-inputs.test.mjs tests/runner-e2e/prepare-report-lock.test.ts`: 4 files, 68 tests pass. - `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --check` reports `changed: false`. - Local run on macOS (darwin-arm64). Full regression is left to CI. ## Risks - Low risk. The change is test data only, and it uses the values that the registry command derives from the committed lockfile and the reviewed `report-lock.json`. - The same two values are in #20770. Whichever pull request merges first, the other one merges cleanly. - The refresh still resyncs these pins only when it commits a lockfile change. If the pins become stale with no lockfile change, the refresh does not repair them, and the support suite shows the failure and the fix command. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, file editing). Extended reasoning was used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 07:26:18 -08:00
DottaandClaude Opus 5.5	e2cf9a670f	fix(runner-e2e): resync SDK public lock pins and make the lockfile bot keep them current (#20623 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner support suite proves the plugin SDK public types from pinned sources in `tests/runner-e2e/source-sdk-public-inputs.json` > - That registry pins the committed `pnpm-lock.yaml` blob and a `lockSha256` that must equal the `outputSha256` in `tests/runner-e2e/report-lock.json` > - The lockfile bot refresh #20615 changed the lock and `report-lock.json`, but the bot does not write the SDK registry > - The bot merges without CI, so the drift guard from #20579 could not block it, and every PR now fails `ci / Runner support suite` > - The guard also checked only the source blobs, so it did not name the `lockSha256` drift, and the sync script could not update `lockSha256` > - This pull request regenerates the registry, derives `lockSha256` from the reviewed policy, and makes the bot resync and check the registry in its own commit > - The benefit is that master CI is green again, and the next lockfile refresh cannot leave the SDK proof stale ## Linked Issues or Issue Description Refs #20615. Refs #20579. Refs #20550. Refs #20613. Overlaps #20618, which refreshes only the registry data. This PR has the same data change and also closes the gap that let the drift land. What happened? After #20615 merged, `ci / Runner support suite` failed on master and on every PR (for example runs 42877750315 and 42877750957, and PR #20613). `source-sdk-public-type.test.ts` and `source-sdk-evidence.test.ts` failed with "Reviewed SDK reference policy does not match public proof". `source-sdk-public-inputs.test.mjs` failed with "pnpm-lock.yaml (changed)". Expected behavior A lockfile refresh keeps the SDK public proof registry in step with the lock and the reviewed report-lock policy. If the registry drifts, the failure names the stale pin and the fix command. Steps to reproduce 1. Check out master at `3bf7af6c77`. 2. Run `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk`. 3. See the "Reviewed SDK reference policy does not match public proof" failure. ## What Changed - Root cause: #20615 changed `report-lock.json` from output `2360d5e9…` to `b3208306…` and changed the `pnpm-lock.yaml` blob from `851e65af…` to `6d825f97…`. The registry kept the old values. `refresh-lockfile.yml` commits only the lockfiles and the report-lock proof, and it merges without CI. - `source-sdk-public-inputs.json`: regenerated with `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --write`. Only `inputs["pnpm-lock.yaml"]` and `lockSha256` change. The compiler, parser and Zod pins do not change. - `source-sdk-public-inputs.mjs`: the sync now copies `lockSha256` from the committed `report-lock.json` `outputSha256`. It refuses when the committed lock is not that policy's reviewed input or output. This is the same rule that `prepareReviewedSdkReference` applies. Only the lockfile bot can write that policy, so a contributor cannot choose `lockSha256`. - `staleSdkPublicInputs()`: also reports a `lockSha256` that differs from the committed report-lock output. The guard test then names both stale pins and the `--write` command. - `source-sdk-reviewed-fixture.ts`: the "Reviewed SDK reference policy does not match public proof" error now names the two values and the fix command. The check itself is unchanged. - `refresh-lockfile.yml`: after the lock commit, the bot runs the SDK sync with `--write`, amends the commit, and runs `--check` before it pushes. `head_sha` is still read after the amend, so the bot merges the exact commit it checked. - Tests: new sync tests for a lockfile refresh and for an unreviewed lock, and a workflow-order test in `lockfile-refresh-merge.test.mjs`. - Docs: `doc/RELEASE-AUTOMATION-SETUP.md` §8.1 and `tests/runner-e2e/OPERATIONS.md` describe the resync. ## Verification - Reproduced on master: `source-sdk-public-type.test.ts` and `source-sdk-evidence.test.ts` failed with the policy mismatch. - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk prepare-report-lock sdk-canary workflow-security`: 9 files, 135 tests pass. - Before the registry change, the extended guard reported `pnpm-lock.yaml (changed), tests/runner-e2e/report-lock.json outputSha256 (lockSha256 changed)` with the fix command. - `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --check` reports `changed: false`. - `node --test .github/scripts/tests/lockfile-refresh-merge.test.mjs .github/scripts/tests/lockfile-refresh-workflows.test.mjs .github/scripts/tests/lockfile-trust.test.mjs`: 39 tests pass. - `pnpm test:e2e:runner:unit` on a host with load near 28: 6 files timed out, and none of them are the SDK proof files. All 15 of those files pass when run alone. CI runs the full suite. ## Risks - The bot step now runs one more repository script. If that script refuses (for example, an unreviewed committed lock), the refresh PR is not pushed. Before this change, the bot merged a refresh that made master red. - `lockSha256` now comes from the reviewed policy, not from the old registry value. The proof check is unchanged. It still needs the lock bytes to hash to `lockSha256`. - If #20618 merges first, this branch rebases cleanly, because the registry change is identical. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit) and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-11 06:57:51 -08:00
github-actions[bot]andlockfile-bot	a4e89288ec	chore(lockfile): refresh pnpm lockfiles (#20824 ) ## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description What happened? Dependency manifests on master changed after the last committed lock refresh. Expected behavior Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. Steps to reproduce Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>	2026-11-11 14:28:26 +00:00
DottaandPaperclip	556b249897	fix: preserve exact Markdown in native document writes (#20605 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents save Markdown documents through native tools and REST routes. > - The native write_document handler trims the supplied body. > - This removes final newlines and can change Markdown whitespace. > - This pull request preserves the original body after required-content validation. > - Agents can now write and read the same Markdown string across these paths. ## Linked Issues or Issue Description What happened? The native write_document tool removes leading and trailing whitespace. A body with one final newline loses that newline on read-back. The REST document PUT and write_task_document preserve it. Expected behavior The native write_document tool must preserve the supplied Markdown body exactly. Required-content checks and key and title normalization must stay in place. REST-backed paths retain surrounding whitespace and keep their existing escaped-line-break conversion for single-line payloads. Steps to reproduce 1. Write a disposable document through write_document with a final newline. 2. Read the document and compare the full body with the supplied string. 3. Repeat through write_task_document and REST. These paths preserve the string. Related work: Refs #18824. That change aligns document keys. This change preserves body strings. A search of open and closed PRs found no duplicate fix. ## What Changed - Use a separate untrimmed required-string helper for the document body. Keep the existing requiredString callback signature and all other field normalization. - Add real tool-handler, database, and authenticated HTTP tests for create, update, read-back, and replay. Cover leading spaces, hard breaks, blank lines, final newlines, no final newline, Unicode, and mixed line endings. - Keep blank and invalid bodies rejected. Document whitespace preservation and the existing REST escaped-line-break exception. - Fix the Runner support archive-scanner shutdown defect exposed by CI: rejected ZIP scanners now receive SIGKILL so an ignored SIGTERM cannot leave publication waiting for process closure. Keep all secret and archive limits unchanged. A real child-process regression fails before the change and passes afterward. - Address all three initial review findings: callback type compatibility, contribution comment policy, and the REST documentation limit. ## Verification Final tested head: `c9ff8cf3f83aa19e5d8313c65816a3a2e6562368`. - All applicable [CI checks](https://github.com/paperclipai/paperclip/actions/runs/43019885807) passed. This includes typecheck, build, Runner support, Rust, native Runner, general tests, serialized suites, all eight E2E shards, and the required aggregate gates. - [Greptile review](https://github.com/paperclipai/paperclip/runs/130520170309) passed on this head with confidence 5/5. All three review threads are resolved. The independent Paperclip review also reports 5/5 with no findings. - Red at `7368a0b15c2f02107778a961144d86e2eb59a952`: `pnpm --dir server exec vitest run src/services/native-runtime/task-document-body.integration.test.ts` failed because write_document removed the final newline. Cross-task and REST cases passed. - Green on the final source: 176 tests passed across the document body, document key, native tool authority, task scope, document service, and document route suites. The tests cover create, update, exact read-back, replay, blank rejection, revision guards, and company scope. - The ZIP shutdown regression failed before repair with “scanner did not stop”. After repair, 54 tests passed across zip-inspection-shutdown, private-reporter-artifacts, manual-private-cell-report-assets, and private-evidence-budget. The regression uses a real process that ignores SIGTERM. - Earlier CI runs had two archive-related timeouts. The source repair above replaces blind reruns. The Runner support suite and aggregate verify gate now pass. - A disposable live native document reproduced the original whitespace loss. REST preserved the same string. The fixed handler passed local integration tests. The running service has not been deployed. - Local environment: Linux x64, Node 25.9.0, Vitest 5.0.3, and embedded PostgreSQL. Local Rust preparation was unavailable because cargo is absent. Final-head CI passed Rust verification. `pnpm verify --explain` requested broad CI because evidence.ts is a global input. Required CI supplied that coverage. ## Risks Low risk. The change retains required-content validation. The product change affects document body whitespace handling. The CI repair forcefully stops only already-rejected archive scanners; it retains fail-closed publication and all existing scan limits. It does not change schema, authorization, revision guards, receipts, or persistence. ## Model Used CI repair continuation: GPT-6.1, using repository tools and code execution. OpenAI Codex, GPT-6 (the exact serving model ID and context window were not exposed in this session). Used reasoning, repository tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-11 06:38:32 -06:00
Nicky Leach	4d141cdf64	Carry plugin task refusals and batch project reads (#20762 )	2026-11-11 00:49:03 -08:00
DottaandClaude	9574065a28	fix(workspaces): hand on a proven-unchanged preparation epoch instead of abandoning it (#20736 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native run on a remote sandbox first restores its workspace's saved version, publishes that root as its dirty epoch, and only then starts work > - A restart that kills the run in that window leaves the epoch dirty. The ownership settle (`settlePreparationOnlyRemoteWorkspaceEpoch`) always abandons it: the root is forgotten and the retry restores the saved version into a new directory > - The saved provider session still names the old directory, which is what #20712 and #20723 now have to handle > - In the common case the killed preparation only restored the saved version and changed nothing. A durable proof for exactly that already exists (`proveRemoteWorkspaceUnchanged`, used when the next admission finds an ended writer) > - This pull request makes the settle use that proof first, and keeps abandoning whenever the proof fails > - The benefit is a stable workspace root across a restart during preparation, with no change for anything unproven ## Linked Issues or Issue Description Refs #20712 What happened? In restart-chaos waves, a server kill during a follow-up's preparation (right after the workspace restore published) made the ownership settle abandon the epoch. The retry restored into `<workspace>-<epoch>-<lease>` instead of the published root, and the inherited session checkpoint named a root that no longer existed. Expected behavior When durable records prove the killed preparation only restored the saved version, the workspace is handed on at that saved version and keeps its root. Anything unproven is abandoned exactly as before. Steps to reproduce 1. Run a remote native task to a saved version, then start a follow-up or continuation run. 2. Kill the server after the follow-up's restore publishes (crash point `remote_workspace.prepare.after_publish_before_provision`). 3. Let ownership reconciliation settle the dirty epoch, then let the retry prepare. ## What Changed - `recovery/remote-workspace-unsaved-files-settlement.ts`: before it abandons a preparation-only epoch, the settle calls the existing `releaseUnchangedRemoteWorkspaceWriterById` (lock the workspace, check the writer, then `releaseUnchangedRemoteWorkspaceWriter`). That releases the writer at its saved version (dirty false, writer cleared, location and root kept, baseline marked released, run reference set to `saved`) only when `proveRemoteWorkspaceUnchanged` holds. The proof requires all of these durable records: - the unchanged-baseline marker for this exact epoch, run and saved version. It is written in the same transaction that publishes the restored root, so a kill mid-restore leaves no marker; - no recorded process, process group, process start or provider session; - no execution-start run event, no native result and no native runner checkpoint (the controller writes one before every runner command); - no workspace operation other than the file-neutral restore, finalize and recovery receipts (provisioning and saves record theirs before they touch files); - every workspace repository already in the saved version. - If the proof fails for any reason, the settle abandons the epoch exactly as before. The existing settle preconditions (run terminal, still `preparing`, no finalization, no work-started event, no save in flight) are unchanged and run first. - `packages/shared/src/crash-point-registry.ts` and `remote-workspaces.ts`: add the crash point `remote_workspace.prepare.after_publish_before_provision`, so the chaos harness can kill exactly at this boundary. It counts only a preparation that published an unchanged baseline (a restore of a saved version), and it is a no-op unless a test arms it. `tests/runner-e2e/source-sdk-public-inputs.json` is refreshed because the registry file is a pinned SDK input. - Test `ownership-reconciliation-report.test.ts`: a proven-clean epoch is handed on with its root kept, with no abandonment receipt, and the writer hold is free for the retry. These cases are still abandoned exactly as before: no baseline (an epoch saved by the previous release), a baseline for another epoch, an already-released baseline, a provisioning operation. A run with a recorded process is still neither handed on nor abandoned by this settle. Previous-release story: an epoch saved by a release that did not write the baseline marker has no proof, so it is abandoned as before. No persisted shape changes; the marker already exists on master. ## Verification - Red: with the release branch disabled, the new test fails on the clean case (the epoch is abandoned). Green with the change. - `cd server && npx vitest run src/__tests__/ownership-reconciliation-report.test.ts src/services/remote-workspace-snapshot-reuse.test.ts src/__tests__/crash-point-call-sites.test.ts` passes. `packages/shared` `crash-points.test.ts` passes. `tsc --noEmit -p server` passes. - Live Daytona waves (claude, workload S1, the new crash point at the continuation's preparation (`:2` before the point was gated to saved restores; `:1` now), image built from master `e039be9202`, one task each): \| \| before (release disabled) \| after \| \| --- \| --- \| --- \| \| Killed epoch 2 \| abandoned (`workspace.remote_epoch_abandoned`) \| handed on (`workspace.snapshot_skipped_unchanged`, epoch 2) \| \| Retry root \| new root `…-4-ecf1c81e…` \| same published root `…-1-33e93b90…` \| \| Retry `task.prepare` \| 4598 ms \| 4731 ms \| \| Retry restore / environment acquire \| 1141 / 1381 ms \| 1106 / 1478 ms \| \| Task result \| done, no Retry \| done, no Retry \| \| Stopped or orphan sandbox at rest \| none \| none \| Preparation time does not change in this setup. The Chaos Lab environment uses ephemeral leases, so every run gets a fresh sandbox and a full restore with or without this change, and the killed run's sandbox is destroyed either way. The gain is the stable root, which keeps the saved session's working directory valid. ## Risks - Low. The release path is the same one admission already uses for an ended writer, under the same proof. The settle now tries it first. - A proof false positive would keep a root whose files changed. The proof refuses on every record that precedes a write, and on any missing or older-release evidence. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-11-11 00:33:18 -06:00
DottaandClaude Opus 5.5	cbd636d652	fix(evals): wait for rendered replies, bound runs by role and grade chat outcomes structurally (#20685 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat-feature eval catalog runs real chat journeys and grades saved state, rendered links and run ledgers. > - The second full-cohort run on master `9296c74e8d` failed 15 cases where the product did the right thing and the harness did not. > - The causes were three harness classes: links read before the reply rendered, raw run counts that rejected product-caused follow-up runs, and keyword graders on free-form replies. Four browser locators also targeted retired controls. > - This pull request fixes those harness layers. It does not change the product. > - The benefit is that these cases measure the feature again: correct behaviour passes, and the relevant broken behaviour still fails. ## Linked Issues or Issue Description What happened In the second full-cohort chat-eval run on master `9296c74e8d`, these cases failed although the reply and the saved state were correct: `agents.choose-primary`, `cases.case-history`, `budgets-costs.set-budget` (links read about 2s before the reply rendered); `task-coordination.independent-review`, `cases.update-case-facts`, `routines.manage-other-owner`, `questions-and-approvals.review-plan` (a legitimate continuation or the product's completion follow-up exceeded a raw run count or was cancelled); `cases.cases-disabled`, `inbox-search.find-design`, `navigation-preferences.broken-link`, `cases.browse-cases` (keyword or clause graders rejected correct replies); `agent-chat.conference-room`, `workspaces.inspect-git`, `task-coordination.block-on-design`, `skills.author-skill` (browser steps used stale settings or controls). Expected behavior A correct reply and correct saved state pass. A wrong outcome, extra or duplicate agent work, a substitution or a missing route still fails. Steps to reproduce Run `runner-full-stack-e2e.yml` with `cadence=manual`, `all=false` and an `id` list of the cases above against master `9296c74e8d`. Read `result.json` and `snapshots/` in each cell's evidence. Refs #20600 ## What Changed - `chat-rendered-links.ts`: `collectConversationLinks` waits (at most 10s, once per reply per page) for the latest saved lead reply to render when its conversation is open. A page that does not show the conversation does not wait. `observeInternal` and the flow conversation read use it. - `budgets-costs.set-budget`: waits for the rendered reply before it reads anchors. It fills the budget only after the form shows the saved cap, and checks that the typed value stays. - New `chat-catalog-domains/run-roles.ts`: identifies a run by role (a worker woken by a comment that a named run saved on the task; the CEO's chat completion follow-up). - `task-coordination.independent-review`: admits one Morgan continuation woken by the CEO's authorization comment and one Riley review run. The review request may come from that continuation's native status decision. Other runs still fail. - `cases.update-case-facts`: a phase settles when its CEO turn and at most one Morgan continuation woken by that turn's task comment finish. A new `role-bound-runs` check rejects any other run. `terminal-counts` gets an optional `maximumTerminalStatusCounts`. - `routines.manage-other-owner`: admits one Morgan run on a task that the CEO created for Morgan. A schedule change must be Morgan's own write from that run. The ownership check is structural (a linked owner task, or a link to the routine or Morgan's page). The keyword checks on the reply are removed. - `questions-and-approvals.review-plan`: after the stale-review check, lets the product's completion follow-up settle before final run custody. The existing grader already validates it. - `cases.cases-disabled`: the negation-blind false-claim regex is removed. The case passes with a reply link to the Experimental settings page (the scripted person follows it and sees the Cases control) or a pending choice card. Saved state still proves that nothing was created. - `inbox-search.find-design`: the per-clause regex is removed. The first task link in the reply must be the task with the approved revision. - `navigation-preferences.broken-link`: a link to the similar task fails only when its label names the original (identifier or exact title). A separate mention passes. - `cases.browse-cases`: an excluded case must not be a result row. When the reply has no result list, every case link is a result, so an excluded case must not be linked. - Browser steps: `conference-room` reloads after the fixture enables the flag. `inspect-git` and `block-on-design` open the panel with "Show properties" and pick the tab from the side panel launcher. `author-skill` does not repeat the full inspection in `verifyResult` when `act` already inspected the same saved version. - Each changed case has a new `oracleVersion`. `chat-catalog-descriptors.json` is in sync. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts <119 affected test files> --testTimeout=180000`: 1673 passed, 3 skipped. One file (`chat-factual-query-input.test.ts`) did not load in the local worktree because `@paperclipai/plugin-sdk` is not built there. It does not import changed code. CI runs it. - `chat-catalog-consistency.test.ts` passes (242 descriptors). - `npx tsc --noEmit -p tests/runner-e2e`: no errors in changed files. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: no pin change. - `node --test` on the three `chat-full-catalog-testops-*.test.mjs` files: 40 passed. - Calibration against retained evidence: the retained replies and run shapes of cases-disabled, browse-cases, find-design, broken-link, independent-review, update-case-facts and manage-other-owner now pass. Broken variants fail (closed case as a result row, earlier design linked first, original label on the similar task's link, unprompted or duplicate worker runs, a second review run, a routine change outside the owner's run, no route link). - Paid runs on this branch: see the PR comments. ## Risks - Low risk. Only eval harness code changes. - The structural graders do not read reply wording. A reply that words the outcome wrongly but links the right records can pass. A person reads the reply. - `navigation-preferences.sidebar-order` is not changed. The shipped sidebar has no per-project order or collapsible Agents section, so the case premise needs an owner decision. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 20:24:41 -06:00
Nicky Leach	0d4b1d8c89	Gate agent plugin tabs by execution environment (#20657 )	2026-11-10 15:04:44 -08:00
Nicky LeachandPaperclip	f30ac226d6	Keep project environment mappings independent of shared defaults (#20651 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A project supplies work context and repository storage to task runs. > - Creation selected a storage owner but did not retain the selected environment on the project. > - Configuration stored an environment inside checkout policy, but policy reads and run selection dropped it. > - Projects therefore continued to follow shared agent and instance defaults. > - This pull request saves placement on each project and uses it for new runs. > - The benefit is an independent Project to Environment association with the existing storage, trust, and recovery safeguards. ## Linked Issues or Issue Description Refs #20459 and #11644. Follow-up to #20587. The reported journey is Project > Configuration: changing one project's environment must not change another project's mapping or shared defaults. Related open PR #18176 repairs placement through the legacy workspace-policy field. This follow-up introduces a separate project association, saves creation defaults, and migrates saved legacy references. It overlaps that PR's placement, permission, and recovery safeguards. The changes should be reviewed together. #9588 concerns environment-variable response privacy and does not address placement. ROADMAP.md has no separate project-association work planned. ## What Changed - Add nullable `projects.environment_id` and expose `Project.environmentId`. Save the effective creation default once on the project. Keep environment placement separate from checkout/isolation policy. - Read and write Environment directly in Project Configuration. Keep the control available with isolated workspaces disabled. Each form reads its project's value, even when environment choices share a query cache. - Offer environments compatible with the current storage owner. Change the target and storage together when the existing storage selector attaches a core project to a plugin. Preserve retained-data restrictions and validate the observed storage owner again on save. - Resolve new task runs from their organizational project's target before agent and instance defaults. File sources from another project do not change placement. Explicit low-trust sandbox choices and enforced sandbox policies retain precedence. External provider placement remains separate. - Restore admitted native runs from their owned environment lease. Later project edits apply to new admissions. Require workspace-write permission for environment changes and clears. Check company bindings before runtime admission and during lease creation. - Migrate valid saved policy references into the project column. Normalize legacy API writes. Count and clear matching project references when an environment is deleted. Update API and behavior documentation. - Refresh the maintained SDK source registry for the two changed shared files. Runtime pins are unchanged. ## Verification - [Hosted CI](https://github.com/paperclipai/paperclip/actions/runs/42908045146) is green on `8b3b7d5b5aed51fc4c8e9c2e3506508bbaf9e00c`: 60 successful checks, three skipped gates, no failures, and `ci / verify` passed. GitHub reports no merge conflicts. Standard Greptile scored the same commit 5/5 with both review threads resolved. - Source checkout on macOS arm64, Node 26.12.0, with fresh isolated PostgreSQL. No production data or live provider calls. - The two-project component test changes the first form with isolation disabled and a shared query cache. The second form keeps its target. Storage changes update only the selected project. A saved target remains selected while the list loads, and an explicit clear remains possible. - API integration creates two projects from the same default, patches one, reads both, and checks that agent and instance defaults remain unchanged. A later instance-default change does not rewrite either saved project target. Tests cover unavailable/incompatible choices, anonymous and cross-company denials, and storage preconditions. - The migration test runs the actual SQL against an existing project row and verifies its environment and disabled checkout policy survive. Deletion accounting and clearing affect only matching project references. - The final guard set passes 53 tests across four files: `project-environments.test.ts`, `environment-selection-route-guards.test.ts`, `heartbeat/environment-selection.test.ts`, and `ProjectProperties.environment.test.tsx`. All 64 environment-run-orchestrator tests pass, including direct lease acquisition without an extra caller argument. The broader affected set passed 109 tests across eight files before the additional admission guards. These sets overlap. - UI typecheck, migration numbering/safety, token gates, module boundaries, query-error rendering, and diff checks pass. `pnpm verify --explain` returns exit 2 and requires broad CI because shared schema/runtime paths and behavior documentation changed. Broad coverage is delegated to current-head CI. - Early iterations failed due to a legacy parser that drops the environment field, a fixture without real company membership, and incorrect expectations for privacy-preserving 404 denials. The service now reads legacy references before normalization, the API fixture has real membership, and denial assertions match the existing privacy contract. The orchestrator mock also needed the reused company-binding guard in its service contract. The focused suites above pass after those corrections. - Local server typecheck remains blocked by installed ACPX, Slack, AI SDK, and Runner type mismatches in unchanged dependency-facing code. It reports no errors in the changed production paths. This is not a full local typecheck or suite pass. Hosted CI supplies full current-head coverage. - SDK source-proof and privacy qualification passes 56 tests across three files, including the real built SDK. The maintained source registry matches the committed revision; runtime pins are unchanged. - Review reproduced two failures: an incoming policy without a storage owner could be validated before the existing owner was restored, and uppercase legacy UUIDs missed migration matching. Preserve the owner before validation and compare normalized UUID text. Both regressions now pass. The post-fix project/storage/orchestrator batch passes 107 tests: `node node_modules/vitest/vitest.mjs run server/src/__tests__/project-environments.test.ts server/src/__tests__/project-workspace-provider.integration.test.ts server/src/__tests__/environment-run-orchestrator.test.ts --silent`. - The first hosted run failed build, typecheck, and canary at the same `run-completion.test.ts` call because lease acquisition required a new argument. The lease method now enforces binding internally. Its existing caller compiles without a new field, and the orchestrator tests prove the binding flag reaches the runtime. The full server check no longer reports that error; the local dependency errors above remain. ## Risks - Apply the normal database migration before serving the new revision. The migration adds a nullable foreign key and copies valid legacy project selections. It does not move repository data or resize provider storage. The five newest schema snapshots are retained. - New projects retain their creation target when shared defaults change. Existing projects without a saved selection continue to inherit defaults. A null target explicitly opts into inheritance. - Existing saved project selections now affect placement. Missing, archived, incompatible, and foreign-company targets fail validation; they do not silently choose another storage owner. A running admitted process keeps its original owned lease. - Local dependency limitations are listed above. No live-provider, deployed-browser, or production rollout qualification is claimed. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact serving snapshot and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 14:31:48 -08:00
DottaandClaude Opus 5.5	353945c86d	fix(tool-gateway): let agents create gateway tokens without receiving the value (#20613 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Tool gateways let external MCP clients use a company's tools with a gateway client token > - Since delegated connection management (#17737), an agent can create and revoke gateway client tokens for its responsible person > - #20581 stopped agents from getting the one-time token value, but it did this by making token creation person-only > - The product decision is that agents keep their delegated authority to create and revoke tokens > - This pull request restores agent token creation, but the agent never receives the token value: the person reveals the value once on the gateway's Tokens page > - The benefit is that an agent can do the rotation work for the person, and the secret still never enters chat, run events, logs or tool output ## Linked Issues or Issue Description Refs #17737 Refs #20581 What happened After #20581, `POST /api/tool-gateway/gateways/{gatewayId}/tokens` returns 403 `gateway_token_requires_person` to every agent caller. Agents lost the delegated authority to create gateway client tokens that #17737 gave them. Before #20581, an agent that created a token received the plaintext value in its tool result and could repeat it in chat. Expected behavior An agent with delegated connection management can create and revoke gateway client tokens. The agent never receives a token value. The responsible person gets the value once, on the gateway's Tokens page. Person callers keep today's behavior. Steps to reproduce 1. Give an agent delegated connection management for a responsible person with `tools:admin`. 2. As the agent, call `POST /api/tool-gateway/gateways/{gatewayId}/tokens` (for example through `call_api`). 3. See 403 `gateway_token_requires_person`. ## What Changed - Agent create: `POST /api/tool-gateway/gateways/:gatewayId/tokens` from an agent with delegated authority returns 201 with token metadata only (`id`, `tokenPrefix`, `expiresAt`, `pendingReveal: true`), plus `tokensPath`, a `reveal_credential` `requiredUserAction` for the responsible person, and a `nextStep` instruction. The saved row has no usable value: its stored hash is of a value that is generated and then discarded. Gateway authentication also rejects `pendingReveal` rows. - Person reveal: new person-only `POST /api/tool-gateway/gateway-tokens/:tokenId/reveal`. It generates the value, stores its hash, clears `pendingReveal` and returns the value once. A conditional update makes a second or concurrent reveal return 409. Only the token's responsible person can reveal it (403 `gateway_token_reveal_requires_responsible_person`). Agent callers get 403 `gateway_token_requires_person`. A token revoked before reveal cannot be revealed. - Why a reveal that generates the value: no stored plaintext existed to reveal, and storing one (even encrypted) for later display adds a secret at rest. Generating the value at reveal time keeps the existing one-time display and stores only a hash. - Schema: `tool_mcp_gateway_tokens.pending_reveal boolean not null default false` (migration `0460_gateway_token_pending_reveal`). Shared `ToolMcpGatewayToken.pendingReveal`, and `ConnectionRequiredUserAction` gets `reveal_credential` / `gateway_token`. - UI: the Tokens page shows a "Waiting for you to reveal" section for pending tokens with a "Reveal token" button. The value appears in the existing "New token — copy now" one-time block. Pending tokens show the status "Not revealed". - Agent gateway create: the 201 for agent callers also includes `gatewayPath`, `tokensPath` and a `nextStep` that tells the agent to link the saved gateway. - The agent create-token `nextStep` and the catalog say not to revoke the old token in the same turn: its client keeps working on it until the person installs the new value. - The reveal update and its `tool_gateway.token_revealed` activity entry commit in one transaction. The Tokens page offers Reveal only for tokens that act for the signed-in person. - API reference: new `docs/api/tool-gateway-tokens.md` (create, reveal, revoke) in the docs nav. - Revoke stays allowed for agents with delegated authority (unchanged route). - API catalog: create-gateway, create-token, reveal and revoke descriptions now say who acts, that agents never receive values, the page paths, and that the old token is revoked only after the client works with the new one. - Agent access map: create-token policy note and a new reveal entry. - Evals `gateways-profiles.create-gateway` and `gateways-profiles.rotate-token`: - Remove the keyword grader (`human\|operator\|admin\|manage`) and the handoff text checks. Grade saved state, rendered links and secret absence. A plain reply or a question card is accepted. - Remove the setup probes that called the create routes as the agent. Under delegated authority, those probes created a real gateway ("Forbidden agent gateway"), which failed create-gateway before the agent ran. - create-gateway: exactly one gateway for the intended profile and company, created by the agent (or by the person after a handoff); the person opens it from a rendered link; attribution matches the creator; any agent-created token is still pending reveal; no token value in chat, cards or run events. - rotate-token: after the agent's turn, both setup tokens still work and any replacement the agent created is pending the person's reveal; the person follows the rendered Tokens page link and reveals (or issues) the value in a private browser; the new value works before the old token is revoked; the unrelated client keeps access; no token value appears anywhere, including a regex scan for any full `pcgw_<id>.<secret>` value. - The changed scenario context and outcomes are recorded in `chat-full-catalog-testops-manifest.json` `changedFields` and pinned in the manifest test. The inventory digest, descriptors (`oracleVersion` v2, `sourceFiles`) and SDK public input pins are updated. The unused `external-gateway-handoff.ts` module is removed. - Fixture realism: the create-gateway profile now allows a real read tool (an empty profile made the agent ask which apps to add instead of finishing), and the rotation tokens get a neutral owner note instead of "Disposable rotation fixture" (the agent read that note as an eval hint). - Shared private-browser helpers (`external-private-browser.ts`, `external-private-browser-worker.ts`) get one optional `pendingTokenId` input and one `revealedPending` receipt field. ## Verification - Red: on `master` route code, the new integration tests `lets agents create and revoke gateway client tokens without ever receiving a token value` and `reveals an agent-created gateway token value only to its responsible person` fail (403 `gateway_token_requires_person`). - Green: `cd server && npx vitest run src/__tests__/connection-management.integration.test.ts -t gateway` (25 passed). The test checks that the agent `call_api` result has no `token` field and no token-shaped value, that the agent cannot reveal, that the person reveals once (second reveal 409) and the value authenticates, that agent revoke works and stops the value, that a token revoked before reveal never gets a value, and that person create still returns the value. - `npx vitest run src/__tests__/tool-gateway.test.ts src/__tests__/openapi-routes.test.ts src/services/native-runtime/runner-api.test.ts` (2247 passed). - `cd ui && npx vitest run src/pages/apps/gateways/panels/TokensPanel.test.tsx src/pages/apps/gateways/gateway-helpers.test.ts` (16 passed); `pnpm check:token-gates` clean. - Eval graders and catalog: `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` on the gateway oracle tests, `external.test.ts`, catalog consistency/contract, TestOps export/retained, phase/clock callers and SDK input/type tests (all pass); `node --test` on the TestOps manifest, adapter, contract, streaming and reconcile tests (all pass); `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check` (unchanged). - `node scripts/check-maps.mjs` passes. `tsc --noEmit` for `server`, `ui` and `tests/runner-e2e`: no errors in changed files. - Paid end-to-end run of both eval cases, run [42872921890](https://github.com/paperclipai/paperclip/actions/runs/42872921890) on `39f38a7f6c` (before the last three commits): both FAILED, for agent-behavior reasons the later commits address. Not rerun yet. - rotate-token: the agent created the replacement (pending reveal, no value), then revoked the old token in the same turn, despite the create response saying to wait. Its reasoning cited the eval-looking fixture note. The grader rejected the premature revocation. The reply linked the Tokens page correctly. - create-gateway: the agent created the gateway and one pending token, then asked a question card about which apps to add to the empty profile, with no link to the gateway. The grader rejected the missing link. - Token values: a scan of all evidence files of both cells found no full `pcgw_<id>.<secret>` value and no `pcgw_<uuid>` form. Agent `call_api` payloads in run events are redacted; the integration test proves the agent tool result has no value. ## Risks - Behavior change for agents: token create returns metadata and a Tokens page path, not a value. Agents that expect `token` in the response must send the person to the Tokens page. - Migration adds one boolean column with a default. Existing tokens get `false` and keep working. - A pending token that is never revealed stays as an unusable row until someone revokes it. It cannot authenticate. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 15:13:34 -06:00
Devin FoleyandPaperclip	8148fc0b1e	fix(release): fetch draft-notes PR summaries in batches (#20602 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release workflow drafts the next stable's release notes when a beta publishes (`draft_stable_notes`) > - The draft nests each referenced PR's summary under its commit subject, and it fetches each summary with its own `gh pr view` call > - A beta range now references more than 1,500 PRs, so these serial calls do not finish inside the job's 10-minute timeout > - This pull request fetches the summaries in batches of 50 per GraphQL query > - The benefit is that the job finishes in under a minute and pushes the notes branch again ## Linked Issues or Issue Description What happened? The `draft_stable_notes` job of [run 42763789753](https://github.com/paperclipai/paperclip/actions/runs/42763789753) (beta `2026.1110.0-beta.0`) reached its 10-minute timeout. GitHub cancelled it during "Draft stable notes from the published beta", while a `gh` process was still running. The beta itself published and its smoke tests passed. The `release-notes/v2026.1110.0-beta.0` branch was not pushed. Expected behavior The job writes the notes draft and pushes the branch for every beta, whatever the size of the commit range. Steps to reproduce 1. Publish a beta whose range from the last stable has many PR-referencing commits. This range has 1,586 commits, and 1,583 different PRs. 2. `draft-stable-notes.sh` calls `gh pr view <n>` once for each commit, one after another. 3. The job reaches `timeout-minutes: 10` before the loop ends. Paperclip version or commit `09e24dd12c36251dd12af2d73dbc715ce990f922` (beta `2026.1110.0-beta.0`). This was the first beta to run the job after #12124 added the PR summaries; earlier betas skipped it. ## What Changed - `scripts/draft-stable-notes.sh`: - Collects the unique PR numbers first. Fetches their bodies 50 per `gh api graphql` query (aliased `pullRequest(number:)` fields), into a temporary cache that `enrich_pr` reads. - Parses the raw response with `jq`, not with `gh --jq`. When any number in a batch is not a PR (for example, an issue reference), gh exits non-zero and does not apply its `--jq` filter. The other PRs in the batch still have usable data. - The output format does not change. A missing `gh` or `jq`, a failed batch, a cache directory that `mktemp` cannot create, or `DRAFT_NOTES_SKIP_PR_ENRICHMENT=1` still gives bare subjects. - No code comments are added (per `CONTRIBUTING.md`); this description holds the explanation. - `scripts/draft-stable-notes.test.mjs`: - The enrichment tests now use a fake `gh` that answers GraphQL like GitHub. - New tests: batching (120 PRs make 3 queries, and a repeated PR is fetched once); a batch with a non-PR number; a failed batch; a failed `mktemp`; prose-only and empty bodies; the opt-out. - Range tests now run with enrichment disabled, so an authenticated `gh` on the test machine cannot reach GitHub. - `package.json`: adds the test file to `test:release-registry`. These tests did not run in CI before. ## Verification - `node --test scripts/draft-stable-notes.test.mjs`: 17 pass. They also pass with macOS bash 3.2 first on `PATH`. - `node scripts/ci-profile/node-test.mjs scripts/draft-stable-notes.test.mjs scripts/release-lib.test.mjs`: 31 pass. - Red check: the new tests against the old script fail 5 cases (all batching and enrichment cases). The `mktemp` test fails against an unguarded `mktemp` call. - Live run on the real range: `./scripts/draft-stable-notes.sh 2026.1110.0-beta.0` finished in 47 seconds. It wrote 4,596 excerpt lines for 1,583 PRs. - Parity: for 40 randomly sampled PRs from that range, the new excerpts are byte-identical to the old per-PR `gh pr view` output. - `shellcheck`: no new findings. The two remaining notices were there before this change. ## Risks - Low. The change touches only the post-publish draft job, and every failure still gives the bare-subject draft. - `jq` is now needed for the PR summaries. GitHub's Ubuntu runners include it. Without `jq`, the draft has no summaries, the same as without `gh`. - I did not change `release.yml`. The job already checks out `master`, so after this merges, re-running the failed job of run 42763789753 should push the missing notes branch with the fixed script. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) via Claude Code, with tool use (shell, file editing, test runs, read-only GitHub API calls). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 11:48:55 -08:00
DottaandClaude Opus 5.5	5f34f777ec	fix(evals): grade 16 chat cases by saved state instead of reply keywords (#20600 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat-feature eval catalog checks that a chat agent does the work a person asks for > - In the latest full-cohort run on master, 16 cases failed although the saved work was correct > - Their graders matched words, one literal value, or one API route in free-form model output > - A grader that fails correct work hides real failures in the noise, and it can also pass wrong work that uses the expected word > - This pull request grades those cases by saved state, audit records, setup literals and rendered page structure, and keeps only narrow checks for false claims > - The benefit is that these cases fail only when the product or agent does the wrong thing ## Linked Issues or Issue Description Refs #18971 (related: company-secret question handoffs) What happened The full-cohort chat-eval run on master failed 16 cases where the retained evidence shows correct behaviour. Examples: `companies.rename-company` accepted only `company.updated`, but the supported branding route logs `company.branding_updated`. `recovery.post-stop-message` required a signup button label that matched a regex, and rejected a labelled email form with the button "Start writing free". `secrets.secret-proposal` flagged "the value is delivered when Morgan next runs" as a false delivery claim. Other graders required words such as "uncommitted", "telemetry", "administrator" or the app name that the question did not ask for. Expected behavior A grader accepts every supported route and wording that achieves the outcome. It fails when saved state is wrong, when a requested fact is wrong, or when the reply makes a claim that saved state contradicts. Steps to reproduce 1. Load the retained observation for one of the cases below from the full-cohort evidence. 2. Grade it with the old oracle. The keyword or route check fails although the saved state is correct. 3. Grade it with the new oracle. It passes. A broken variant (wrong saved state or a contradicting claim) still fails. ## What Changed - `activity.audit-change`: attribution is graded by structure. The reply names the recorded editor, names no other company agent (new setup literal `otherActors`), and states only real transitions. Sentence grammar is no longer matched. - `app-permissions.ask-before-action`: each continuation reply must exist and must link no app record other than the reviewed connection. It no longer must name the app. - `budgets-costs.spend-breakdown`: the reply must point at the record with unresolved usage (task link, identifier or run id) and must not give it a zero or free amount. The words "subscription" and "incomplete" are no longer required. - `cases.browse-cases`: an excluded case may be mentioned in prose. It must not be linked or listed as a result row. - `cases.handoff-same-task`: handoff instructions carry findings forward when they point at the saved `findings` document, reproduce it, or restate it. The premise (a primary Morgan run performs the handoff) is unchanged. - `cases.update-case-facts`: a saved browser value may add clauses that only rule out other platforms ("desktop Safari unaffected"). - `companies.agent-observability`: the reply no longer must name Telemetry and the run log. It must not claim that they are exported. - `companies.rename-company`: both supported name routes (`company.updated`, `company.branding_updated`) are accepted. - `connection-setup.choose-provider`: the answer must exist and must not claim a connected account. The unavailable method no longer must be named; the setup path stays graded by clicking the card or link. - `goals.align-existing-work`: a question card from the answering run counts as the phase-two response. Refusal words and "unchanged" words are no longer required; false change claims still fail. - `recovery.post-stop-message` (and `steering.interrupt-with-direction`, which shares the helper): a visible form with a labelled field and a submit control is a signup action, whatever the button says. - `routines.pause-owned`: the "paused"/"running" words are no longer required. The saved checkpoint proves both; a cancellation claim still fails. - `secrets.secret-proposal` (and `secrets.configure-company-secret`, which shares the helper): a sentence that says when delivery will happen is not a delivery claim. - `skills.explicit-restriction`: the reply no longer must say "administrator". Success claims and invented tool limitations still fail. - `workspaces.inspect-git`: the reply no longer must say "uncommitted". Unchanged HEAD and status prove it; commit or push claims still fail. - Every affected `oracleVersion` is bumped in code and in `chat-catalog-descriptors.json`. No scenario YAML changed. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts --testTimeout=180000 tests/runner-e2e/chat-catalog-domains tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts`: all pass. - `PAPERCLIP_CHAT_EVAL_RENDERED_ARTIFACT_CANARY=1` rendered-artifact canary with Chromium: pass, including new form cases. - `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs`: pass. `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: unchanged pins. - `npx tsc --noEmit -p tests/runner-e2e`: no errors in changed files. - Offline calibration on retained evidence, for each case: the retained correct evidence passes, earlier passing evidence passes where it exists, and a broken variant fails. `spend-breakdown` retained evidence still fails, because its reply shows unresolved usage as `$0.00`. `handoff-same-task` retained evidence still fails on the kept primary-run premise. - Live paid runs: see the comment below. ## Risks - Low risk. The change is limited to eval graders and their unit tests. Product code does not change. - Each grader is more accepting of wording. Saved-state checks and the narrow false-claim checks remain, so a wrong outcome still fails. - `secrets.configure-company-secret` still fails its retained evidence. Its agent asked a two-question card, and the act supports only one-question cards. This pull request changes it only by the shared delivery-claim helper. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 13:37:16 -06:00
DottaandPaperclip	89ec4e7614	feat(links): add cached GitHub pull request previews (#20593 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task conversations contain links to pull requests and other work. > - A URL alone cannot tell the operator whether a PR is open, merged, or blocked by conflicts. > - Fetching every link at render time wastes API calls and can expose private data through a shared cache. > - This pull request adds a demand-driven links service and a GitHub PR adapter. > - Each viewer and company has a separate cache. Repeated links share one request and result. > - Operators get context when they hover or focus a link. Normal links stay useful when a provider is unavailable. ## Linked Issues or Issue Description Subsystem affected Task Markdown links, shared link metadata, and the server GitHub API boundary. Problem or motivation PR links have compact labels and neutral icons. They do not provide a rich hover card or a common status source across repeated mentions. A background resolver cannot borrow a person's GitHub connection because it has no viewer identity. Proposed solution Recognize supported links locally. Request metadata after a delayed hover or keyboard focus. Show a card at the first line of the link, aligned with its icon. Share the result across all mentions in the current viewer and company. Keep the exact original navigation URL. The first adapter supports GitHub pull requests. Open is green, merged is purple, and merge conflicts are orange. Draft, closed, unknown, loading and unavailable states remain quiet. Alternatives considered Eager page scans would spend API quota on links the operator does not inspect. Company-wide caching of personal GitHub results would cross the credential boundary. This change uses neither approach. Roadmap alignment ROADMAP.md was checked. This extends existing task links and the GitHub resolver. It adds no deployment adapter or new background worker. Searches of open and closed PRs for link previews and PR hover cards found no matching implementation. ## What Changed - Add host-owned link recognition and a normalized preview contract. The service selects an adapter by provider. - Add a board-only company read endpoint. It checks company access before cache reads and sends a private, no-store response. - Use the current viewer's eligible GitHub connection. A configured connection that cannot supply credentials fails closed. With no managed connection, reuse existing company credentials or public access. Instance credentials retain the public-repository check. - Reuse the GitHub external-object resolver for API transport and response parsing. Add title and mergeability to its normalized snapshot without changing its existing state contract. - Coalesce requests by company, viewer and canonical PR URL. Bound cached entries and concurrent work. Cache success for one minute and failures for a short period or the provider retry window. - Add an abortable request deadline. Timed-out work stays in the concurrency budget until it settles. No periodic worker or idle polling is added. - Add a shared browser query and anchored preview card. Repeated Markdown links observe the same result. There are no automatic retries, window-focus refreshes, toasts or connection prompts. - Add ten production-component Storybook examples and real pointer/keyboard browser checks. Document the API and verification route. ## Verification - Linux source checkout: 233 focused tests passed in nine suites. They cover recognition, fixed API destinations, provider states, access failures, credential selection, viewer/company isolation, request coalescing, expiry, cache bounds, concurrency bounds and mounted OpenAPI coverage. - A live public GitHub smoke returned this PR's correct title and open state. Two equivalent URLs shared one real network request. Personal connection access was not part of that smoke. - A controlled HTTP server stalls a real response body. The test proves cancellation while another lookup completes. - UI typecheck and the server layered typecheck passed. The UI production build passed. - Token gates and query policy gates passed. All 30 map/catalog support checks passed. The three new server test files also passed a focused TypeScript check. - All 27 focused browser checks passed on the reviewed source tree. They cover light/dark themes, narrow layouts, open/merged/conflict/draft/closed/loading/unavailable/failed states, fixed positioning during pointer movement, exact navigation and keyboard access. A real browser HTTP interception counts one deferred lookup for two equivalent PR links. - The first browser attempt lacked Chromium. A later combined build/test attempt exhausted the sandbox's 8 GiB memory limit and killed Storybook. Those attempts supply no pass evidence. Chromium is installed, and the final browser checks run with the build stopped. One initial Storybook dependency reload interrupted a streaming check. All eight existing inline-link cases passed on the warmed server, followed by a complete 27-case pass. - Synthetic Storybook pointer events did not exercise the native hover boundary. The assertions now run through real Playwright pointer and keyboard actions. The fixture stories retain the production components. - Review fixes preserve the Draft label when GitHub reports both draft and conflicts. They also preserve neutral link colors in dark mode. The 25 affected service checks pass, including the added combined-state regression. Browser checks compare unknown, draft and closed links with the muted theme color. - The first CI policy run rejected a literal retry opt-out. The code now uses the existing explicit zero-retry helper. A new test-spy type error was also corrected. These failures are fixed; their earlier runs remain failed. - `pnpm verify --explain` returned exit 2 because the browser test path selects broad CI. No full local suite is claimed. The final CI run supplies that coverage. - The previous head `437913c797963aa113bb82e2ae8b836969317ef9` passed CI and received a 5/5 Greptile score. All four review threads are resolved. Master then advanced. The current head `ae2a1c2adbd61a621cca9bdd160561952613df6e` resolves the source-manifest conflict against master `9b21dcbdda01296349f3cfe0a3a6891e79df6130`. Fresh CI and review are running. - The previous CI run found a lost class on decorated PR links and a stale SDK source manifest. Both are fixed. The existing CommentThread integration test and 26 focused link service tests pass. All 45 SDK source/privacy tests pass. All eight source-manifest helper tests pass. The manifest helper changed only source identities and added the two new shared files. Runtime, compiler, parser and library pins stay unchanged. - A later review found that expired timeouts could reuse a settled promise. The service now removes settled requests from its shared-request map and counts unfinished provider work separately. The new regression failed before the fix and passes after it. The concurrency-bound and real stalled-HTTP tests also pass. - UI typecheck and token gates pass after these fixes. The browser rerun needed Chromium in the new session. That first attempt did not exercise the UI. The complete 27-case browser rerun passed before the master merge with retries disabled. All 27 browser cases also passed on the merge revision with retries disabled. - After the master merge, all 84 focused link, OpenAPI and integration tests pass. All 55 SDK source/privacy/manifest tests pass. UI typecheck and token gates pass again. The source manifest retains master's reviewed runtime pins and includes both sets of source changes. GitHub confirms the PR can merge cleanly. The preflight helper rebuilt the shared package and plugin SDK; Cargo is unavailable locally. Final CI supplies Rust coverage. - The latest master update changed the reviewed report-lock policy but left the SDK lock hash at its previous value. The SDK source/evidence checks reproduced the mismatch. This change sets only the SDK lock hash to the authenticated output of the existing master policy and patch. Compiler, parser and Zod pins stay unchanged. All 56 SDK source/evidence/manifest tests now pass. All 84 focused link/API/integration tests, UI typecheck and token gates also pass on the updated source. ## Risks - This cache is process-local. Separate server replicas can make separate GitHub requests. - A cached result can remain visible for its short lifetime after GitHub changes. A later hover refreshes expired data. - GitHub can report unknown mergeability. The UI shows conflicts only when GitHub explicitly reports them. - Personal preview data stays in viewer/company caches. It is not written to company external-object records or persisted browser query storage. - Stories simulate the API. They do not prove a live GitHub login or private-repository access. - No schema migration is required. ## Model Used OpenAI Codex, based on GPT-6. The runtime does not expose the exact serving model ID or context-window limit. Used reasoning, repository inspection, source editing, shell execution and browser tools. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 19:32:51 +00:00
DottaandClaude Opus 5.5	42d4499489	fix(daytona): force a stuck stop before starting a retained sandbox (#20609 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On Daytona, a follow-up task resumes the agent's retained sandbox before the agent starts > - The host stops an idle sandbox gracefully. A process that ignores SIGTERM can leave Daytona in `stopping` for many minutes > - Resume waited for that state change until its 300 s start deadline. It never escalated, so each attempt failed after five minutes, and the bounded retries did the same > - On staging, a user comment waited about 21 minutes before the agent started. Two resume attempts each used 299.7 s in the plugin dispatch > - This pull request forces a stuck stop once, then starts the sandbox within the same deadline. It also traces the wait with the observed state > - The benefit is that a stuck stop costs about 30 s instead of failing the run, and the next case is diagnosable from the trace ## Linked Issues or Issue Description What happened? A user commented on a task that used a retained Daytona sandbox. The first run and its first retry each failed after about 5 min 20 s with `reusable_sandbox_resume_failed (cause=provider_error)`. The Honeycomb trace shows `execution.plugin_dispatch` at 299.7 s in both. Inside it there is only `lease.lookup` (44 ms) and `lease.channel_cleanup`, and no `lease.start`. So the whole budget went to `waitOutSandboxTransition`, which polled a sandbox that stayed in a transition state. The later attempt, after a control-plane restart, found the sandbox `stopped`. Its `lease.start` took 1.3 s. A transition that ends in `stopped` is a stop. Expected behavior A sandbox that stays `stopping` for longer than the host's own graceful-stop budget is forced to stop, then starts. The run does not fail on the start deadline. Steps to reproduce 1. Retain a Daytona sandbox whose graceful stop does not finish (Daytona reports `stopping`). 2. Resume or realize the lease. 3. Before this change, the call polls until the 300 s deadline and fails with "is still stopping at its start deadline". ## What Changed - `packages/plugins/sandbox-providers/daytona/src/plugin.ts`: `waitOutSandboxTransition` tracks how long the sandbox stays `stopping`. After `STUCK_STOPPING_FORCE_AFTER_MS` (30 s) it calls `sandbox.stop(seconds, true)` once, then keeps polling. A refused force stop keeps the old wait within the same deadline. - The wait is a `lease.wait_transition` span. The force stop is a `lease.force_stop` child span. - `lease-tracing.ts`: the closed `paperclip.sandbox.startup.state` label also records `starting`, `stopping`, `restoring`, `archiving` and `resizing`. Before, these were all `other`. - `server/src/services/plugin-host-services.ts`: the host admits the two new span names and the new closed state values. - `doc/observability.md`: documents the new spans and states. ## Verification - Red/green: with the plugin change stashed, the three new stuck-`stopping` tests (realize, refused force stop, resume) fail by hanging to the deadline. With the change they pass. The "settles on its own" test confirms that no force stop happens for a normal stop. - `vitest run` in `packages/plugins/sandbox-providers/daytona`: 14 files, 618 tests passed (8 skipped). `tsc --noEmit` passed. - `server/src/__tests__/plugin-host-services-span.test.ts`: red before the allowlist change, 25/25 after. - `pnpm verify` exited 2 (the doc is a global CI input), so CI supplies the broad coverage. - Not verified on a live Daytona sandbox. Hosted acceptance needs the change deployed to staging. ## Risks - The force stop sends SIGKILL to processes in a sandbox that is already stopping and is about to start. The disk persists across a stop, so the retained workspace is kept. A graceful stop that is still running past 30 s is killed. The host's own graceful stop budget is 30 s, so the host had already given up on it. - Daytona can refuse a stop while a state change is in progress. Then the code waits as before, so the result is no worse than today. - The root cause is inferred from the trace (a transition that ends in `stopped`). The new state label confirms or refutes it the next time it happens. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, browser, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 19:20:30 +00:00
DottaandClaude Opus 5.5	9b21dcbdda	refactor: remove SPEC docs and repoint spec citations (#20565 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors and agents read the docs under `doc/` before they change code > - `doc/SPEC.md` and `doc/SPEC-implementation.md` were out of date. `doc/SPEC-implementation.md` became an append-only log: in 60 days it got about 1,645 new lines and lost 181, across 140 commits > - #20557 changed `AGENTS.md` so the code and its tests are the source of truth, and the specs are only background > - This pull request removes the two spec docs. It moves only the sections that live code comments cite into focused docs, and it repoints all other references > - The benefit is one less stale document that agents read and append to ## Linked Issues or Issue Description Refs #20557 (merged). This PR is the follow-up that removes the docs. It is based on `master`, which already contains #20557. Issue type Outdated content. The docs are out of date and conflict with the code. Where is the issue? `doc/SPEC.md`, `doc/SPEC-implementation.md`, and the code comments and docs that cite them. What's wrong? `doc/SPEC.md` and `doc/SPEC-implementation.md` are out of date, but agents still read them and append a paragraph with each fix. Suggested fix Delete both files. Move the sections that code cites into focused docs. Repoint the references. ## What Changed - Deleted `doc/SPEC.md` and `doc/SPEC-implementation.md`. - Moved these spec sections, because live code comments cite them: - §9.3.1 "Shared default-open issue writes" → new section in `doc/agent-permission-defaults.md`. - §9.8.1 "Issue-thread interaction resolver contract" → `doc/execution-semantics.md`, "Issue-thread interaction resolution". That section already had most of the contract. I added only the missing parts: the `supersedeOnUserComment` default, the canonical policy list, the addressee and cap rules, the rule that surfaces show the server's effective audience, and the cross-issue budget charge. The short policy summary at the top of the section now points to the list, so the definitions occur one time only. - §9.9 "Task Watchdog Authority Contract" → new "Authority contract" section in `doc/TASK-WATCHDOG.md`. It holds the allowed and disallowed mutation lists and the deferred capabilities. The atomic recovery batch, restoration verification, and interaction rules were already in `doc/execution-semantics.md` §11, so the new section links there and does not repeat them. I did not move the "Downstream acceptance criteria" checklist or the sentence about the CTO and Security sign-off. These were implementation-time notes; the tests now hold that coverage. - §9.10 "Company Skill Policy Contract" → new `doc/company-skill-policy.md`. No focused doc had this contract. I did not move the "Required regression tests" phase checklist. - Did not move any other part of the specs. - Repointed code comments that cited spec sections: - `packages/shared/src/issue-write-denial.ts` (§9.3) → `doc/agent-permission-defaults.md` - `server/src/routes/issues.ts` (§9.9) → `doc/TASK-WATCHDOG.md`, "Authority contract" - `server/src/routes/issues.ts` (§9.3, §9.8.1 cross-issue budget) → `doc/agent-permission-defaults.md` - `ui/src/lib/interaction-audience.ts` and `ui/src/components/InteractionGovernancePanel.tsx` (§9.8.1) → `doc/execution-semantics.md` - `ui/src/lib/skill-policy-denial.ts`, `ui/src/pages/CompanySkills.tsx`, `ui/src/pages/CompanySkills.production.tsx`, `ui/src/pages/SkillStudio.tsx` (§9.10) → `doc/company-skill-policy.md` - `server/src/services/recovery/ownership-invariant-report.ts` (monitor deadline wait path) → `doc/execution-semantics.md` §8. That section already lists the one-shot monitor as a valid wait path. - Resynced the git blob pin for `packages/shared/src/issue-write-denial.ts` in `tests/runner-e2e/source-sdk-public-inputs.json`. The comment edit changed the file's hash, and the Runner support suite checks that pin. I used `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --write`; only that one pin changed. - Updated `AGENTS.md` §1 so it does not name the deleted files. - Removed the links to the deleted files from `doc/plugins/PLUGIN_SPEC.md`. - Updated the `doc-maintenance` agent skill (`.agents/skills/doc-maintenance/`) so it no longer audits or stages `doc/SPEC.md`. - Removed or replaced references in `maps/README.md`, `maps/feature-map/README.md`, `maps/storybook-map/README.md`, `doc/PRODUCT.md`, `doc/CLIPHUB.md`, `doc/DEPLOYMENT-MODES.md`, `doc/DEVELOPING.md`, `doc/evals.md`, `doc/spec/agent-runs.md`, `doc/AGENTCOMPANIES_SPEC_INVENTORY.md`, `doc/connections/CONNECTOR-PLAYBOOK.md`, `doc/plugins/ideas-from-opencode.md`, `doc/execution-semantics.md`, `doc/TASK-WATCHDOG.md`, and `.devin/wiki.json`. - In `doc/execution-semantics.md`, the "admission is independent of cleanup and accounting" reference now states the rule inline: only a hard stop, an explicit pause, or an inactive company refuses admission. - Kept these references as they are: dated plans in `doc/plans/` and `docs/plans/`, the dated log `doc/logs/2026-05-24-cli-api-parity-e2e-log.md`, `releases/`, the transcript fixture `ui/src/fixtures/runTranscriptFixtures.ts` (it uses the filename only as sample text), and `docs/specs/external-task-protocol.md` (it refers to the Symphony project's own `SPEC.md`). ## Verification - `git grep -n "SPEC-implementation\\|SPEC\.md"` (without `releases/`, and ignoring `PLUGIN_SPEC.md` citations) finds only the kept references listed above. - `grep -rn "§9\.$3\\|8\\|9\\|10$" ui/src server/src packages/shared/src` finds no results. - `node scripts/check-maps.mjs`: "Maps aligned: 35 areas, 192 capability IDs". - `pnpm check:token-gates`: "All gates clean". - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/source-sdk-public-type.test.ts tests/runner-e2e/source-sdk-evidence.test.ts`: 45 passed. These failed in CI before the pin resync. - `pnpm verify`: exit 2 ("Broader verification required") because `AGENTS.md` is a global CI input. Required CI supplies the broad coverage for this revision. - All code changes are comments only. No runtime behavior changes. The title uses `refactor:` because comments in source files changed. ## Risks - Low risk. The change removes docs and edits comments only. - A reader can lose context from spec sections that I did not move. The text stays in git history. The code and its tests are the source of truth. - External links to the two deleted files will return 404. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with tool use (shell, file edits). Standard context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused checks above; CI supplies broad coverage) - [x] I have added or updated tests where applicable (not applicable: docs and comments only) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 13:07:59 -06:00
DottaandClaude Opus 5.5	bce79f186f	fix(evals): count a pending question card as the turn's response where the case allows it (#20597 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat catalog evals check that a chat agent answers the person and saves the right records > - An agent can end its turn on a question card; the product then records `interaction_owns_conversation_response` and posts no reply comment > - The privileged settle step and some graders still waited for a reply comment, so valid card-ended turns timed out or failed > - This pull request adds one shared rule for when a pending card counts as the turn's response, and uses it where the case allows a card > - The benefit is that card-ended turns are graded on what the person saw, while cases that need a written reply stay strict ## Linked Issues or Issue Description Refs #20588 (request_human_input version fix found by the same runs). Refs #20590 (chat review cards link their document). What happened? In the full chat eval run on master `f4337bb5fb`, agents ended turns on pending question or review cards. `developer-labs.component-reference` timed out in the privileged settle step, which required a reply comment. `goals.align-existing-work` refused the foreign goal in a question card, but the grader read only reply comments. `questions-and-approvals.reject-and-revise` returned its draft in a review card bound to the draft revision, but the grader and the result check required a reply link. Expected behavior A pending card counts as the turn's response only when the product decided that the card owns the run's response, the lead's own run created it, a person can answer it, and it is not a governed approval. Each case still grades the content it requires. Steps to reproduce Regrade the retained evidence of the cases above with the old graders: they fail on the missing reply comment although the card is on the chat. ## What Changed - New `chat-catalog-domains/turn-response-card.ts`: `turnResponseCards()` returns pending cards from runs whose `presentationDecision.reasonCodes` contains `interaction_owns_conversation_response`. A card must be created by the lead, be human-answerable (a text question, two or more choices, or an ordinary confirmation) and not carry `toolAction`, `secretProposal` or `connectionAuthorization`. - `settlePrivilegedChat` treats such a card as a settled turn. The privileged graders still require their written guidance, so `developer-labs.component-reference` now fails at grading instead of timing out. - `goals.align-existing-work` reads the card from the run woken by the foreign-goal request as that phase's answer. A leaked foreign text, or a card that does not refuse, still fails. Oracle `paperclip.chat-catalog.internal.goals-align.v2`. - `questions-and-approvals.reject-and-revise` accepts a review card bound to the draft's exact saved revision. The result check opens the draft from the card's document link (needs #20590). Oracle `paperclip.chat-catalog.plan-review.v6` for both plan-review cases. - Descriptors updated for oracle versions and `sourceFiles`. ## Verification - `turn-response-card.test.ts`: the retained cards pass; other run, other agent, answered, other company, user-created, non-human resolver, single non-text choice, governed confirmation, and a run that replied all fail. - Offline regrade of retained evidence (master `f4337bb5fb` full-cohort run): `goals.align-existing-work` cross-company check passes with the card and fails without it; `reject-and-revise` `real-local-review-draft` passes with the bound card and fails with no card or a card bound to another revision. - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-catalog-domains ...clock tests --testTimeout=180000`: 269 files passed (one file needed the plugin SDK build locally, then passed). - Consistency and contract tests pass. `npx tsc --noEmit -p tests/runner-e2e` is clean. SDK public input pins are unchanged. - Paid run [42855262851](https://github.com/paperclipai/paperclip/actions/runs/42855262851) on a temporary branch with this PR, #20588 and #20590 (OpenCode, local): `reject-and-revise` and `revoke-personal-grant` passed; `goals.align-existing-work` failed only `phase-bound-strategic-context-explanation` (keyword check, unchanged here); `developer-labs.component-reference` ended on a question card again and now failed at grading (`existing-question-reference`, `reference-only`) instead of timing out in settle. This run's agents replied in text in the two passing cases, so the card branch of `reject-and-revise` is covered by the offline regrade only. ## Risks - `settlePrivilegedChat` now stops waiting on a card-ended turn. Privileged cases whose graders need a reply fail sooner and with the real failed check; none pass more easily. - Changed oracle versions start new history for `goals.align-existing-work` and both plan-review cases. - The remaining keyword check `phase-bound-strategic-context-explanation` in `goals.align-existing-work` is unchanged and still fails the retained reply; it needs its own fix. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), agentic coding with tool use in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 12:33:52 -06:00
github-actions[bot]andlockfile-bot	9816d2d25c	chore(lockfile): refresh pnpm lockfiles (#20615 ) ## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description What happened? Dependency manifests on master changed after the last committed lock refresh. Expected behavior Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. Steps to reproduce Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>	2026-11-10 18:20:31 +00:00
DottaandPaperclip	12f70d0d4c	Keep Slack form custody and receipt transfer durable (#19825 ) ## Thinking Path > - Paperclip manages work for AI-agent companies. > - Chat connections deliver provider messages and user answers. > - Slack requires a response within its original three-second deadline. > - Forms and their delivery receipts must survive retries and tenant restarts. > - This change uses the existing connector queue and durable form owners. > - Current authorization, exact receipt checks and normal ACK rules stay in force. > - The tenant can settle useful work while keeping the connection enabled. ## Linked Issues or Issue Description Refs #19019 and #19348. Companion Cloud PR: paperclipai/paperclip-cloud#1250. ## What Changed - Preserve form grants, sessions, encrypted evidence, terminal receipts and company-authorized Retry. - Check initial form enrollment before the provider send. Keep a typed preparation wait only while an exact current enrollment obligation exists. - Refund the attempt only when the claimed attempt and credential lease still match and provider invocation has not started. Possible or accepted sends retain uncertain-delivery custody. An ordinary error with the same text still uses the normal five-attempt limit. - Notify the existing publication worker when normal enrollment commits. Reconstruct pending work from durable state after a lost reply or owner restart. - Page settled history before proving idle safety. Keep unfinished, malformed and foreign receipts visible beyond the first 1,000 rows. - Keep Slack callback, answer, receipt and ACK rules in their provider owners. Support body-bound Cloud idle capabilities when a complete catalog exceeds the header budget. - Add focused regression coverage, an enabled quiet/reactivation recipe and explicit fixture port checks. - Keep SDK evidence tests tied to a normally built reviewed reference, with a separate strict refusal for actual unreviewed CI outputs. ## Verification App head: `6284d6881b5dc38a9ee149bcbde35e69a67dd8ca`, including actual master `dff8bed007e01c32f3826144b893ec80603d1335`. Companion Cloud head: `f83b38187dc7e6b2e219fe865dc16d268cc66fdb`, including actual Cloud master `b1dcc3421a91efffee631a61536cf48f357c22fc`. Cloud #1250 is merged at `1d07d81f23d4627ab7b6dcf49176b4128a2f8f9b` after exact-head passing checks and complete Greptile 5/5. Current exact-head CI typecheck, build, tests and aggregate verification are green; the complete edited Greptile review is 5/5 and all 13 review threads and nested comment pages are resolved. - Both latest enrollment findings were reproduced against the preceding product. The pure readiness control returned no wait for the first enrollment. The real-PostgreSQL control observed a provider call while that enrollment was still staged. - The repaired source passes three whole files: 21 readiness controls, 80 real-PostgreSQL milestone controls and 37 tenant-actor middleware controls. All 138 pass with zero excluded or TODO cases. The SQL case reconstructs the worker through seven waits, preserves the attempt budget, recovers a controlled lost enrollment reply, and commits one provider-port call and one exact grant. Replaying the selected attempt does not duplicate them. Authorization, SDK and enrollment-response ports are controlled in that SQL case. - The same files cover ordinary five-attempt exhaustion, accepted and possible sends, stale ownership, malformed bindings, attention states and current actor restrictions. One synthetic mismatched-binding parameter uses generation 2 against current generation 1; it is not evidence for an older-generation control. - The normal server-package typecheck passes at the measured source. It runs the normal Runner TypeScript, protocol, SDK and declaration prerequisites and all seven main App programs. The separate declaration configuration is also recorded. A worker union-narrowing error was found, fixed with an explicit callable/null branch, and verified by that normal compiler. No cast or test relaxation was used. - The normal merge of current master preserves all 78 owned paths and 15,783 unrelated master entries and modes. The three measured test files and their changed feature paths are unchanged. New upstream DB code adds only the normal nonunique concurrent native-session index and its migration classification. Shared and adapter-utils changes alter unrelated prompt or denial wording. The 138-case proof carries for those unchanged behaviors; it is not reported as a rerun after the merge. The normal Shared/SDK ensure, DB build, adapter-utils build and complete server-package check also pass at the current merged head. Their fresh inventory contains 8,143 outputs; canonical SQL and journal copies match. This build did not apply the new index migration. - Earlier real-PostgreSQL retained-history coverage reproduced the missing proof with 1,005 settled rows. Its repair retains the existing two-second/64-query inspection budget, current endpoint bound and exact owner snapshot. The 46 source-bound controls cover unfinished and invalid tails, exact cross-page action references, Retry, rollback and lost receipt responses. - Earlier enabled quiet/reactivation coverage produced ordinary useful work, kept configuration enabled, accepted fresh signed work plus a duplicate, and produced exactly one second useful reply. Its two App-worker windows lasted at least 30 seconds with zero measured PG writes, connection or outbound activity after settlement. That evidence stays tied to its measured revision. A missing worker EXIT marker limits that earlier proof; parent evidence does not replace it. - Earlier three-second callback, terminal ACK, current-question race and complete body-size controls remain source-bound. The 500-owner/1,000-binding body measured 313,875 bytes with a 618-byte JWT; signature tampering failed closed. - Final managed-provider acceptance now passes all five fresh Node boots: A/A, A/AB, B/A rejection, B/AB recovery, B/B final. Two independently configured, challenged, verified, linked and enrolled Slack identities exercise ACK and retirement debt separately. Normal provider factory/materialization delivers the configuration through a controlled external stack-provider boundary. The registered opening returns HTTP 200 in 21 ms within the original 3-second deadline. Two answers persist, normal ACK is accepted, exact retirement and lost acceptance/GC replies recover, and the original canonical snapshot remains unchanged without duplicate provider sends or submissions. - The separate complete committed-preflight case passes. A committed `update_attempted` response is lost before SDK entry; normal service-owner reconstruction after the original lease/deadline recovers the same useful update and one answer without a second trigger. This reconstructs service owners within one App process; the separate five fresh Node boots cover process restart and managed signing-key rotation. - Assertion-time PG socket reports balance 15 opens/15 closes across the five boots and 5/5 in preflight, peak 2 and zero live sockets or unexpected attempts. Worker terminal EXIT markers were missing and are not replaced by parent EXIT markers. These are domain/recovery and assertion-time connection proofs, not a new whole-worker terminal quiet measurement. All known owners/groups closed; independent SQL audits were empty; only successful fixture databases were removed. Earlier failed fixture databases/logs remain retained. No product guard, deadline or assertion was weakened. Exact runtime source and normal outputs are unchanged by the three test files in the current follow-up, so these source-bound proofs carry without repeating those journeys. - The preceding App CI ran 11,312 support cases: 11,289 passed, three SDK-evidence cases failed, and 20 external cases skipped. Normal CI dependency setup resolved six stale Slack patch-hash references and produced an unreviewed lock. The public SDK proof correctly refused that lock; the tests incorrectly assumed the CI output was a reviewed reference. The three-file test repair keeps the protected lock policy, source/compiler/parser/Zod pins, scanner and redaction rules unchanged. It builds an owned reviewed reference using the existing normal Shared/SDK producers for the positive seal test, while a separate negative requires actual unreviewed-build privacy refusal and no retained SDK bytes. It does not relabel the CI build as certified or admit a new lock hash. - All four whole SDK support files pass locally: public type 30, evidence 16, public-input synchronizer 8 and lock preparation 10, totaling 64 with zero excluded/TODO cases in 28.73 seconds. The genuine fresh reference build/positive seal and separate invalidated-reference privacy refusal passed within their original 120-second bounds. Original installed dependencies, SDK outputs/cache and runtime source stayed unchanged; normal global setup produced only its own Runner JavaScript outputs. PostgreSQL was denied throughout, all four worker markers were observed, and the owned process group/private test root closed. The local canonical-lock negative mutates only its own reference; fresh remote CI also passes the actual resolved-CI-lock branch: all 16 evidence and 30 public-type tests pass, including the genuine reviewed-reference positive and the strict actual-unreviewed-build refusal. The upstream synchronizer file runs its current 10 cases in CI. - The first complete CI attempt on this head also exposed two separate 30-second timeouts: one real reporter-artifact test and the `agent-test-environment` module-loading hook. Full original logs remain preserved. Their exact test, sanitizer and evidence source blobs are unchanged from the preceding head; inspection found no deterministic setup cause. After the original run finished, one retry of only the failed jobs passed on the same synthetic merge `168851e210671ebe97e7a778e8cf891807e80502`: Runner support reports 11,304 passes and 20 external skips; serialized shard 6 reports all 395 passes, including all 67 route bodies. No source, timeout or assertion changed. This proves recovery; the original causes remain unexplained. - The final pre-merge tree audit uses actual master `17136a6c25ef6cb7c816a29140a4c48c2c75fb31` (refreshed after the earlier `85dec046` audit). The expected normal squash preserves all 81 owned paths and 15,816 unrelated master entries and modes. Only OpenAPI has edits from both sides, and its complete content composition preserves the new project route and token help alongside this PR's API entries. The shared inbound selection changes preserve the same candidate set; new concurrent indexes, Native/workspace behavior and shutdown changes remain upstream behavior. The scoped form proof uses process agents and does not qualify the new Native/sandbox branches. The latest question-schema change still accepts the explicit version-1 payload exercised here. The last master delta also changes managed AI/Native credential refresh and writeback; those branches are outside the measured process-agent form paths. Existing 8,143 local outputs remain tied to their measured revision; no new whole-tree output equality or deployed suspension is claimed. - Reproduce the public path with `doc/testing/slack-cloud-qualification.md`, the companion Cloud checkout, Node 24/22 and fresh owned fixture databases. Aggregate local `pnpm -r typecheck`, `pnpm test:run` and `pnpm build` were not rerun as one command set. Current exact-head required CI and the complete Greptile 5/5 review now pass; the full edited summaries and all paginated threads were read. Local worker quiet does not prove deployed host or Neon suspension. Photon and Discord remain separate work; this PR completes the described Slack slice. ## Risks Use compatible App and Cloud readers before transferring custody. Signatures prove authenticity; current authorization and exact ownership still control execution. Preserve old targets, secrets and terminal facts until their normal owners release them. Keep the managed seal identity stable while custody remains. The required Slack adapter patch remains pinned. Upstream's concurrent index uses the normal top-level migration path. ## Model Used OpenAI `gpt-6.1-sol` implemented and tested the changes. A GPT-6-based Codex managing agent reviewed and integrated them. The host does not expose the managing agent's exact model ID or context-window sizes. The agents used code execution and test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, its version and tool capabilities - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public related issues or PRs - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run the local tests listed above and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green for the current head - [x] Greptile is 5/5 with no open findings for the current head - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 12:17:50 -06:00
Nicky LeachandPaperclip	98ce93dd4e	Choose project storage from compatible environments (#20587 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Projects supply repository files to agent runs through core staging or a storage plugin. > - A storage plugin's default claim silently attached new projects even when their agents used a different environment. > - An environment name or driver type does not establish shared storage access. The existing binding and task-driver contracts do. > - This pull request groups creation choices by that contract and selects storage from the creator's effective environment. > - The benefit is a visible, validated choice and a clear error before an incompatible managed workspace operation. ## Linked Issues or Issue Description Fixes #20459. Related: #20156 covers a separate provider-failure diagnostic. The PR search found no other fix for #20459. ROADMAP.md has no duplicate project-creation change. ## What Changed - Show Available in when configured environments require different project storage. Each option names the environments that can use the same storage owner. Hide the selector when all choices are compatible. - Derive compatibility from repository staging and managed binding ownership. Local, SSH, and sandbox environments use core staging. Managed storage requires a registered task driver from the same installed storage plugin. Separate storage plugins remain incompatible. Unknown plugin contracts and the fake sandbox are excluded. - Follow the creating agent's environment, or the person's primary agent, then the instance default and local fallback. Apply managed-only and forced Kubernetes policy. The product has no separate user or company environment-default field today. - Add a company-authorized discovery endpoint and an optional creation `environmentId`. Validate the choice again on save. Store the storage owner without overriding agent run environments. Preserve explicit API storage choices and existing project ownership. - Keep the hidden repository-free private-task project independent of environment availability. Submit the observed storage owner as `expectedStorageProviderPluginId` so a changed contract fails at save without setting a workspace policy. Instance task-isolation defaults stay inherited. - Reject an incompatible managed workspace before a provider operation. Name the storage and selected environment in the error. Add regression tests, API documentation, creation-story fixtures, and minimal corrections to obsolete map text. ## Verification - 102 tests passed across 7 files: project environment integration, installed storage policy, repository persistence, project environment routes, project creation telemetry routes, OpenAPI, and NewProjectDialog. Run with `node node_modules/vitest/vitest.mjs run server/src/__tests__/project-environments.test.ts server/src/__tests__/project-workspace-provider.integration.test.ts server/src/__tests__/project-repositories-persistence.test.ts server/src/__tests__/project-routes-env.test.ts server/src/__tests__/project-goal-telemetry-routes.test.ts server/src/__tests__/openapi-routes.test.ts ui/src/components/NewProjectDialog.test.tsx --silent`. - The review fixes cover both archived-default private-task cases and changed-storage rejection. With both instance isolation flags enabled, a new core repository project keeps a null policy and resolves to an isolated Git worktree. The complete affected set passes all 102 tests. - The CI query-error policy failure was fixed by using the shared query view. The form and shared query-view suites pass 24 tests, including cached choices during a transient refetch failure. `node scripts/check-query-error-rendering.mjs --enforce --summary` reports zero findings. - CI exposed stale SDK source hashes and heartbeat fixtures that paired plugin storage with Local. Refreshed exactly four shared-source hashes with the maintained generator (runtime pins unchanged). All 53 SDK privacy/source-proof tests, including real built SDK output, pass. All 7 corrected managed-storage heartbeat scenarios pass; the fixtures declare the owning plugin's task environment and substitute transport at the existing execution seam. - Shared and UI TypeScript checks pass. The UI production build passes. `pnpm check:token-gates`, `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and diff checks pass. - Tested UI states: compatible choices, incompatible groups, default within a group, manual override, create failure, loading, discovery failure and retry, and an unavailable default. Tests use the production form and existing design tokens. - The integration tests use real isolated PostgreSQL on macOS arm64 with Node 26.12.0. They cover user/agent defaults, storage persistence, policy restrictions, company and authentication denials, and rejection before a provider call. Discovery uses saved configuration and makes no worker or provider requests. - Local verification limits: `pnpm -r typecheck` and `pnpm build` stop in unchanged adapter-utils code because the installed ACPX types lack `inheritProcessEnv`. A direct server check also finds an unchanged AI SDK type-version mismatch. The existing heartbeat test `inherits the instance default environment across companies while preserving explicit agent overrides` times out at line 518 on both this change and unmodified master with the same dependencies. The full `pnpm test:run` attempt passed all six chat shards (1,948 tests), then the Git workspace expiration suite reported 14 failures. I stopped that broad run. Two sampled failures reproduce on unchanged master with the same `unknown_worktree_git_files` refusal; the other expiration failures were not independently compared. A later broad workspace-preparation run also hit workspace validation failures and was stopped; the seven corrected fixture scenarios pass independently. These limits do not establish a full local suite pass. Hosted CI remains required. - Initial missing-package and loopback-denial failures were resolved by repairing isolated dependency links, building the SDK, and running database tests with loopback access. No deployed or live-provider execution is claimed. ## Risks - Creation defaults now follow the effective environment. A plugin's `defaultForProjects` claim alone no longer selects storage. This is the intended behavior change. - Compatibility covers repository project storage. It does not guarantee arbitrary host paths, retained-workspace moves, adapter/image support, credentials, or worker readiness. Matching provider configs cannot prove a cross-plugin storage contract. - A person's primary agent supplies the personal default because there is no separate personal environment setting. An unavailable configured default requires a valid explicit choice or a configuration repair. - No migration runs. Existing projects keep their storage owner. This change does not detach existing managed data or provide a storage migration. - Full local verification has the limits listed above. CI and standard Greptile review are pending. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The exact serving snapshot and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 17:38:31 +00:00
DottaandClaude Opus 5.5	8e332ccb5e	fix(evals): name stale SDK pins and fix four chat-eval harness defects (#20579 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat feature eval catalog (`tests/runner-e2e`) runs real chat journeys against a real server and grades the saved state, the run's tool events and the rendered reply > - The full-cohort run on master `f4337bb5fb` had 12 failures that come from the harness, not from the product or the model > - Six plugin and secrets cells stopped in 7–9 s, before the server started, because the SDK public source pins were stale > - Four more cells failed on harness timing, a first-visit prompt, or a read receipt that ignored a dedicated tool > - This pull request fixes those four harness defects and adds a cheap guard that names stale SDK pins > - The benefit is that these cells now report on the product and the model again ## Linked Issues or Issue Description Refs #19614, #20550, #20453, #20417, #20483 What happened On master `f4337bb5fb`, these chat eval cells failed for harness reasons: - `plugins.configure-plugin`, `plugins.disable-plugin`, `plugins.install-plugin`, `plugins.use-plugin-tool`, `routines.plugin-managed-routine` and `secrets.least-scope-delivery` stopped before the server started. The source SDK privacy scan refused, because `tests/runner-e2e/source-sdk-public-inputs.json` did not match `packages/shared/src`. #20550 and #20362 already resynced the pins on master. The pins are current at `ce1ee9e89f`. - `agent-chat.track-delegated-work` and `navigation-preferences.edit-profile`: the reply was correct and had the right link. The case read the rendered links immediately after the runs settled, before the reply rendered, so it recorded no links. - `inbox-search.archive-done` and `inbox-search.restore-archive`: setup completes five tasks, so the GitHub star prompt from #20453 opened over the chat and blocked the composer. - `status-cards.explain-card`: the follow-up run read the deleted source with `get_task` and received a 404. The reply said so. `apiReadStatuses` counted only `call_api` reads, so the grader recorded no unavailable read. Why the existing pin check did not stop the drift: `ci / Runner support suite` runs the source SDK tests and is part of the required `ci / verify`. On #19614 it passed at 03:43Z on a merge ref from before #20417 (06:12Z) and #20483 (08:14Z). #19614 merged at 14:02Z without a new CI run on current master. After that, the suite failed on every PR until #20550. So the check exists and fires; the gap was a stale green result, and the failure message was opaque (`SDK public source proof unavailable`). Expected behavior - A change to a pinned SDK input without a pin refresh fails with the changed files and the command that fixes it. - Cases read rendered links only after the saved reply renders. - The eval browser is a returning user's browser that has already seen the star prompt. The product does not change. - A dedicated task read tool (`get_task`, `list_task_documents`, `read_task_document`, also when namespaced) counts as a read of the record it names, with the status its receipt reports. Steps to reproduce 1. Run `chat-features.runner-opencode.local.<case>` for one of the cases above on master `f4337bb5fb`. 2. Read `result.json` and `snapshots/chat-catalog-observation.json` in the evidence bundle. 3. See the failures described above. ## What Changed - `tests/runner-e2e/source-sdk-public-inputs.mjs`: add `staleSdkPublicInputs(root)`. It compares the registry with `git ls-tree HEAD` for every pinned input (the roots and build inputs that the pins file lists), and returns each changed, unpinned or removed file. - `tests/runner-e2e/source-sdk-public-inputs.test.mjs`: a test fails when the committed pins do not match the committed tree. The message names the files and gives the `--write` command. The Runner support suite runs it for every `packages/*` change. A synthetic test covers changed, unpinned and removed inputs. - `tests/runner-e2e/chat-catalog-domains/runtime-handoffs.ts` and `identity-profile.ts`: call the existing `waitForRenderedReply` before they read rendered links. - `tests/runner-e2e/chat-returning-browser.ts` (new) and `chat-catalog-flow.ts`: inside the flow's failure boundary, before setup, mark `paperclip.github-star-prompt.seen` in the eval browser. An init script covers later documents of the app origin, and the open page is marked directly. A test checks that the key equals `STAR_PROMPT_SEEN_KEY` in `ui/src/components/GitHubStarPrompt.tsx`. - `tests/runner-e2e/chat-catalog-domains/api-read-receipts.ts`: bind dedicated task read tools by the digest of the `call_api` input that the server builds for them (`taskBridgeApiRequest`). The result must carry the same `apiOperationId`. Other tools, other records, mismatched receipts and unpaired results do not count. - Bump `oracleVersion` in code and descriptor for every case that uses `api-read-receipts.ts` (shared grader change) and for `navigation-preferences.edit-profile`. Add `internal-comment-reference.ts` to the source files of the two cases that now import it. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts` (whole config): 11301 passed; 1 local-only failure in `controller-replacement-smoke.test.ts` (server spawn, no changed imports). - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts --testTimeout=180000 tests/runner-e2e/chat-catalog-domains tests/runner-e2e/chat-catalog-contract.test.ts tests/runner-e2e/chat-catalog-consistency.test.ts tests/runner-e2e/chat-returning-browser.test.ts` - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk` - `node tests/runner-e2e/source-sdk-public-inputs.mjs $(git rev-parse HEAD) --check` reports `changed: false`. - Red checks: with the `f4337bb5fb` registry, the new pin test fails and names 15 inputs. Without the new waits, the two rendered-reply tests fail. A regrade of the retained `explain-card` run events now gives the follow-up run `unavailable: true, live: false` and the first run `live: true`. - Paid runs of the affected cells (`chat-features.runner-opencode.local.`): - Run 42852698879 (head `d54c04d5f0`): `track-delegated-work`, `restore-archive`, `explain-card`, `configure-plugin` and `plugin-managed-routine` pass. I read each reply and saved state. `track-delegated-work` now records the rendered `/COM/issues/COM-2` link. `explain-card` records `laterUnavailable: true, laterLive: false` from the follow-up `get_task` 404. - `plugins.install-plugin` now starts its server (the pin refusal is gone). It fails a later grading check, `truthful-installation-handoff`: the reply sends the person to the plugin settings page but does not name the package. This is outside this PR. - `inbox-search.archive-done`: the star prompt no longer blocks the composer, and the query is sent. The agent then archives the 501 Done tasks one at a time (about 2 s each, 57 in the 240 s window), because only the per-task archive route exists. This is outside this PR. - `navigation-preferences.edit-profile` did not start in that run (the AWS credentials step timed out). Run 42857020103 (head `2f0b80aa84`): pass. The agent write gets 401, the reply links `/COM/company/settings/instance/profile`, and the person follows the rendered link and saves the name. ## Risks - Low risk. Test harness only; no product code changes. - The receipt change counts more reads as record reads. It only adds dedicated reads bound to the exact record id and operation, so a grader that needs a read can pass on a genuine `get_task` read. The affected cases have new oracle versions. - The star prompt is not tested by any chat catalog case. Product e2e tests for the prompt do not use this flow. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 11:01:44 -06:00
DottaandClaude	84007892e9	refactor: move AGENTS.md section detail to docs and rules, add a bug-fixing section (#20585 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - `AGENTS.md` is the instruction file that each human and AI contributor reads before a change > - Pull request #20557 shortened the file, but several sections still held detail that applies to only some changes > - The scoped-rules section told agents to read each matching rule in full, and most changes match five rules > - The core engineering rules mixed three product invariants with process conventions > - This pull request moves that detail to the docs and to scoped rules, and adds a section on how to fix a bug > - The benefit is a 211-line file that states what applies to every change and points to the rest ## Linked Issues or Issue Description Refs #20557 Issue type Outdated or unclear contributor documentation. Where is the issue? `AGENTS.md` at the repository root, and `rules/`. What's wrong? - The scoped-rules section made agents read the full body of each matching rule before a change. The rule bodies total about 47 KB. - Sections 4, 6, 8, 9 and 10 repeated detail that belongs in `doc/DEVELOPING.md`, `doc/DATABASE.md`, a route rule, `DESIGN.md` and `CONTRIBUTING.md`. - Section 4 had a database reset command for `data/pglite`. That directory does not exist. - Section 5 had seven rules. Four of them were process conventions or path-specific detail. - The file did not say how to fix a bug. Suggested fix Make the rule headers the index. Move section detail to the document or rule that owns it. Keep only the three invariants in section 5. Add a bug-fixing section. ## What Changed - The scoped-rules section now tells agents to list the rule headers with one `grep` command. Agents read a rule body only for a design decision in its area, when the description is not sufficient, or when a reviewer cites it. - `rules/README.md` says that each `description` must state the required behavior in one sentence. - `rules/pull-request-review.mdc` has a description that states its requirements. - Section 4 (dev setup) and section 11 (pull requests) now point to `doc/DEVELOPING.md` and `CONTRIBUTING.md`. - The schema change workflow moved from section 6 to `doc/DATABASE.md`. - The API and auth expectations moved from section 8 to the new `rules/api-route-expectations.mdc`. - Section 9 links to `DESIGN.md` for design rules. `AGENTS.md` no longer tells contributors to run `pnpm check:token-gates`. - Added section 10, "Fixing a Bug": reproduce the error, write a failing test, write the fix, see the test pass, and verify the fix. - Section 5 now has three rules: company scope, contract synchronization, and the control-plane invariants. - The company-scope exceptions moved to the new `rules/company-scope-exceptions.mdc`. - Removed the strategic-docs rule. The plan-doc convention moved to the repo map. The artifact rule moved to section 7. - Removed rule 7. Five code comments that cited `AGENTS.md §5.7` now cite `rules/data-paths.mdc`. - Regenerated the SDK public source pin, because one of those comments is in `packages/shared/src/crash-points.ts`. - Added both new rules to `.greptile/files.json`. - The API route rule applies only to routes that read or write company data. Its status code list is a list of common codes. - The company-scope rule lists the sidebar company order as an instance-wide exception. - The `honeycomb-tracing` skill now points to `rules/data-paths.mdc` for the Telemetry workflow. ## Verification - Open `AGENTS.md` and confirm that each link resolves. A local script checked this. - Run `grep -E '^(description\|globs\|alwaysApply):' rules/*.mdc` and confirm that each rule prints a description. - Run `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`. It passes locally. - `packages/shared/src/crash-points.test.ts` passed locally before the rebase onto `master`. - Five source files changed in comments only. No other test suite ran locally. ## Risks - An agent that reads only rule descriptions can miss a detail in a rule body. Reviewers still apply the full rules. - Pull requests and the definition of done are now sections 11 and 12. Sections 1 to 9 keep their numbers. - Dated plan documents under `doc/plans/` still cite `AGENTS.md §5.7`. They are historical records. - Pull request #20565 also changes `AGENTS.md`. The second one to merge needs a conflict fix in section 1 and section 2. ## Model Used - Claude (Anthropic), in Claude Code with tool use. - `claude-fable-5-2`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>	2026-11-10 11:00:31 -06:00
DottaandPaperclip	ce1ee9e89f	fix: stop prompts from encouraging subtasks (#20362 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents read runtime prompts, tool help, skills, and role instructions. > - Several of these texts tell agents to create child tasks as a normal next step. > - One accepted-plan prompt even forbids work on the current task. > - Task parentage is useful, but it does not need a prompt that encourages task creation. > - This pull request removes those prompts and keeps the task APIs available. ## Linked Issues or Issue Description Refs #10642. This change adjusts prompt wording; it does not add delegation-cycle guards. Current behavior Prompts tell agents to create children after plan approval, use children for long work, and create a task just to pass context. Onboarding requires a child even for a single request. Proposed behavior Continue work on the current task. Keep explicit assignments, optional task creation, parentage, dependencies, and approval gates. Use the separate-task recipes when the user requests that structure or when continuing existing work. Reason and benefit Reduce needless task trees without removing subtask support. Keep the patch limited to wording, existing test expectations, and generated catalog data. ## What Changed - Align accepted-plan prompts and remove direct child-creation advice from runtime and error text. - Make task tool descriptions neutral. Keep their fields, defaults, and handlers. - Update the main skill, onboarding, CEO instructions, company templates, specialist workflows, and matching guides. - Keep historical release snapshots intact. Refresh the explicitly seeded schema fixture's catalog digest; the server still verifies historical release snapshot hashes. - Refresh the affected generated catalogs, capability inventory, semantic contracts, and fixture manifest. ## Verification - Final tested head: `c6e304d0c105cdbfbe0b2ab0a233307fcd91a04b`. - All 66 check runs are complete: 60 pass, one is neutral, and five are skipped by CI conditions. Snyk also passes. No checks fail or remain pending. - [PR CI](https://github.com/paperclipai/paperclip/actions/runs/42846226554) passes typecheck, build, server/general tests, all eight E2E shards, Runner support, Runner Rust, and the aggregate verify gate. [Windows](https://github.com/paperclipai/paperclip/actions/runs/42846226546) and [Docker Runner](https://github.com/paperclipai/paperclip/actions/runs/42846226108) also pass. - [Greptile](https://github.com/paperclipai/paperclip/pull/20362#issuecomment-6751419835) and [Paperclip review](https://github.com/paperclipai/paperclip/pull/20362#issuecomment-6751454010) give this exact head 5/5. All six inline threads are resolved. Neither current summary has findings. - Merged master `bf0220d34a156c81853755c3d1365ddc527037de`. Resolved three generated-file conflicts with the canonical generators. - Local verification passes: 473 focused prompt, continuation, UI, catalog, runtime and native-execution tests; Runner TypeScript build; generated metadata; UI token gates; and diff checks. - The prior head `c22018cb0f` failed three SDK evidence tests because two source hashes were stale. Reproduced all three failures locally. The existing generator refreshed only the hashes for the denial-copy file and its test. All 53 affected evidence/source-pin tests then passed, including private-value rejection. No scanner logic, assertions, or runtime dependency pins changed. Current CI confirms the support suite passes. - A prior commit failed the Rust process reaping test. Its cause remains unproven. No Rust code or assertions changed. Later Rust runs pass; this does not establish the cause of the earlier failure. - Local preflight built shared/plugin SDK packages, then stopped because `cargo` is absent. Current-head CI supplies Rust and full build coverage. `pnpm verify --explain` selects broad required CI; full local suites were not repeated. ## Risks - Wording changes guide the model; they do not enforce a ban on subtasks. - Existing installed skill versions and saved custom instructions need their normal refresh process. This PR does not migrate company configuration. - Existing approved task assignments and approval gates remain in effect. ## Model Used - OpenAI GPT-6, through Codex, with reasoning, tool use, and code execution. The exact routed model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 09:58:54 -06:00
DottaandClaude	8d2bf78323	test(runner-e2e): resync the SDK public source pins with master (#20550 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner support suite checks that the plugin SDK's public types come only from reviewed source inputs, pinned by git blob hash in `tests/runner-e2e/source-sdk-public-inputs.json` > - #19614 landed those pins, generated against an older master; since then `package.json` and ten files under `packages/shared/src` changed > - So `source-sdk-public-type.test.ts` (2 tests) and `source-sdk-evidence.test.ts` (1 test) fail the `ci / Runner support suite` check on every PR merge ref > - This pull request regenerates the source pins with the script made for that job > - The benefit is a green Runner support suite for every open PR again ## Linked Issues or Issue Description What happened? Since #19614 merged, `ci / Runner support suite` fails on every PR (for example #20047 and #20538) with `SDK public source proof unavailable` and `Source SDK evidence privacy check failed`. Steps to reproduce 1. On master at `902f376796`, run `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk`. 2. Two tests in `source-sdk-public-type.test.ts` fail. Expected behavior The source pins match the committed tree, and the suite passes. ## What Changed - `tests/runner-e2e/source-sdk-public-inputs.json`: source pins regenerated with `node tests/runner-e2e/source-sdk-public-inputs.mjs `902f376796` --write` (11 changed blobs, plus new source files). The script reports `runtimePinsChanged: false`: the lock, compiler, parser and zod pins are unchanged. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts source-sdk`: 5 files, 68 passed, 1 skipped (before: 2 failed). ## Risks - Test registry only, no product code. The next change to a pinned input under `packages/shared/src` or `packages/plugins/sdk/src` will need the same resync, as before. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com>	2026-11-10 08:33:32 -06:00
Dotta Claude Opus 5.5 Paperclip	f4337bb5fb	test(evals): land the chat catalog eval harness on master (#19614 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The chat-feature evals measure whether agents complete real chat requests through the product > - The chat catalog eval harness and its repairs live on the `chat-eval/master-candidate` branch, not on master > - Today's work made 139 of 242 chat evals pass in verified single-case live runs. Each product fix landed on master as its own pull request > - The eval-side changes (setups, graders, fault seams, evidence capture) still exist only on the candidate > - This pull request lands the harness on master. Master stays the source of truth for product behavior > - The benefit is one tree: evals run against master commits, and product fixes reach the evals as soon as they merge ## Linked Issues or Issue Description Refs #19441, Refs #19599, Refs #20270, Refs #20334, Refs #20342, Refs #20344, Refs #20354, Refs #20360, Refs #20373, Refs #20382, Refs #20409 What existing behavior does this improve? The chat catalog eval harness and the product it evaluates live on a long-running branch outside master. Eval fixes do not reach master, so master-based eval runs use old setups and graders. Subsystem affected `tests/runner-e2e` chat catalog harness, plus a small set of product changes that exist only on the candidate. Proposed change Land the harness on master. Every product fix from this eval work already merged to master as its own pull request. Conflicts keep master's product behavior. ## What Changed - Most of the diff (808 of 851 files) is under `tests/runner-e2e/`: chat-catalog domains, scenarios, descriptors, the TestOps manifest, private evidence tooling, and their tests. - Graders now check saved state, setup literals and rendered or clicked links. Many keyword and regex checks on free-form replies are removed. Each changed grader has a bumped `oracleVersion` and calibration tests (a valid reply, a reworded valid reply, and broken variants that must fail). - The shared link helper `hasLink` no longer treats a list link (for example `/COM/issues?view=mine`) as a link to every task without a URL key. - Setup fixes build real state through supported product routes: routine history, agent grants through the board grant route, budget holds with a real cap, and a storage-layer archive fault for `inbox-search.archive-partial-failure`. - Browser boundary journeys (sign-in, chat-disabled, onboarding) now write the standard pass evidence files. - Retargeted cases, approved by the maintainer: `agent-chat.chat-disabled` (Conference Room gate), `agents.restore-instructions` (no saved instruction history), `access.manage-membership` (member permission edits stay board-only). - The private evidence trusted files (`manual-private-handoff.ts`, `manual-private-retention.ts`, `manual-private-report-assets.ts`) are byte-identical to master. - Product files outside `tests/` (43 files, +1,328 −49) carry candidate-only changes that master does not yet have. Most are tests. The main code changes are in `request-human-input.ts`, `heartbeat-run-summary.ts`, `native-completion-feedback.ts`, `native-run-finalizer.ts`, `openapi.ts`, `secrets.ts` and `IssueChatLongThreadPerf.tsx`. ## Verification - `pnpm vitest run chat-catalog-domains chat-catalog-contract.test.ts chat-catalog-consistency.test.ts unit-launcher.test.mjs --testTimeout=180000` in `tests/runner-e2e`: 269 files pass, 3,550 tests pass. - `node --test chat-full-catalog-testops-manifest.test.mjs`: 10 pass. - `node tests/runner-e2e/source-sdk-public-inputs.mjs <HEAD> --check`: no change. - `tsc` for `server` and `tests/runner-e2e`: clean. - Live single-case verification: 139 of 242 cases passed on paid runs, and a person read the saved evidence for each. These are single-case runs, not one full-cohort run. ## Risks - Large diff. Most of it is test-only, but the 43 non-test files change product code paths. Review those files first. - Some graders are now less strict on reply wording. A person must read replies for meaning on those cases. - 42 cases need disposable outside accounts (Notion, Slack, GitHub, Daytona and others) and stay blocked. `access.invite-teammate` stays failing until local agent isolation lands, because the runner's durable state keeps a live invite token. - The paid workflow compares the trusted private-evidence files with master. They are unchanged, so this risk is low. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution, in Claude Code. Subagents used the same model. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-10 08:02:11 -06:00
Nicky LeachandClaude Opus 5.5	ff4ba7d319	perf(runner): run launch artifacts in place from a trusted read-only root (#20428 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Rust Runner (`paperclip-runnerd`) starts provider sidecars and proxies from qualified launch artifacts: a Node runtime and the scripts it runs > - Since #12699, Runner copies each artifact into a sealed memfd and checks its SHA-256 on every verification. This defends against a path swap or an in-place rewrite between the check and the exec > - On a hosted agent computer (4 vCPU), the copy and digest of the 127 MB Node takes about 0.26–0.30 s warm and about 1.2 s right after a wake. This is on the path from `session.open` to the sidecar spawn, and again on every warm `run.attach`. Each Runner also holds about 125 MiB of shmem for the copy > - On that computer, the artifacts live on the Runner ROM: a read-only erofs image that the deploy pins by digest. The threat that the memfd defends against does not exist there. On local hosts the artifacts are writable, so the memfd must stay > - This pull request adds an explicit, path-scoped, default-off `--trusted-launch-root <dir>`. Runner runs an artifact in place only when its open descriptor proves that it is on an immutable mount under that root. Everything else keeps the sealed, hashed snapshot > - The benefit is a shorter `session.open` and `run.attach` on agent computers, and about 125 MiB less shmem per Runner, with no change where the flag is absent ## Linked Issues or Issue Description What existing behavior does this improve? Launch-artifact verification in `paperclip-runnerd` (`qualified_launch.rs`). Its callers are the ACPX `verified_transport` (every `session_config`, including the park rebind) and the OpenCode and Cursor Cloud proxy launches. Subsystem affected Rust Runner (`packages/paperclip-runner/runner/crates/runner-core`). Current behavior Every verification opens the artifact, copies all bytes into a sealed memfd, and hashes them. Node is 127 MB. The cost is described in the Thinking Path. Proposed behavior When the host passes `--trusted-launch-root /opt/paperclip/runner` and the root passes its checks, Runner holds each admitted artifact open in place and executes it through `/proc/self/fd/N`. It does not copy or re-hash it. Without the flag, nothing changes. Reason and benefit See the Thinking Path. Refs #12699 (introduced the sealed memfd). Refs #20423 (overlap, see below). ## What Changed - `qualified_launch.rs`: new `TrustedLaunchRoot` and `set_trusted_launch_root` (process-global, set once at startup, like `--task-memory-cgroup`). - Root checks at startup. The path must be absolute, already canonical (no symlink or `..`), a directory, and on a read-only, exec-permitted erofs or squashfs mount. If a check fails, runnerd logs a warning and keeps today's behavior. - Per-artifact checks, on the open descriptor. After today's checks (regular file, not a symlink, not group- or world-writable, open-time identity), the descriptor must be a regular file on the root's device. Its kernel path (`readlink /proc/self/fd/N`) must lie under the root by path components, so `/opt/paperclip/runner2` does not match. Its mount must pass `fstatvfs` (`ST_RDONLY` set, `ST_NOEXEC` clear) and `fstatfs` (erofs or squashfs magic). If any check fails, Runner logs the reason and uses the sealed snapshot. - Admitted artifacts become `VerifiedProcessArtifact::in_place_on_trusted_root` (crate-private). The held file goes through the existing `inherited_artifact` path, so the exec is `/proc/self/fd/N` and `PAPERCLIP_VERIFIED_RUNTIME_EXECUTABLE` still names a descriptor that nested launches can reopen. - New `verify_assigned_executable_launch_artifact` never uses the root. `stdio_mcp.rs` now calls it, so host-assigned stdio MCP binaries always keep the sealed snapshot. - `paperclip-runnerd.rs`: parse `--trusted-launch-root <dir>`. Older runnerd builds ignore unknown flags, so a host can pass it before every Runner understands it. - The trusted path is Linux-only. On macOS and Windows, `TrustedLaunchRoot::open` always refuses. - `tests/runner-e2e/SECURITY.md`: documents how the Kimi and Pi provider-death fixtures behave under a trusted root. ### Decision: the trusted path drops the digest The trusted path does not hash the artifact, not even once per process: - The ROM image is pinned by digest at deploy time. - On the agent computer, the Launcher computes the expected `--acpx-sidecar-command-sha256` and `--acpx-sidecar-script-sha256` from the same ROM files immediately before it starts Runner. A re-hash of an immutable file can only repeat that result. - One hash per process would not help `session.open`, because each task starts a new Runner process. - The mount check is the guard: erofs and squashfs have no write path, so the bytes cannot change while the descriptor is open. ### Fixtures that assert the sealed memfd - Untrusted-mode assertions are unchanged. The fixtures do not pass the flag. - Kimi (`kimi-provider-fault.py` `checked_parent_descriptor`): with a trusted root, the parent descriptor is the ROM's Node file (one link). The existing linked-descriptor branch handles it: same inode as `/proc/<pid>/exe`, and the complete bytes hashed against `nodeSha256`. That branch never accepts the file as a memfd. The existing test `test_linked_descriptor_keeps_regular_single_link_checks_without_memfd_claim` covers it. - Pi provider-death admission binds `/proc/self/fd/3\|7` to the native distribution snapshot's Node. The TypeScript ACPX closure copies that file whether or not a trusted root is set, so the flag does not affect it. ### Overlap with #20423 #20423 (draft) moves the same copy and hash to Runner boot and reuses the result. Where this flag is set (Cloud agent computers), there is nothing left to precompute, so #20423 is mostly redundant there. #20423 still helps hosts without a trusted root (local and sandbox Runners). This PR does not change #20423. ## Verification All runs are on Linux (aarch64, Rust 1.97.1, in a container): - `cargo fmt --manifest-path runner/Cargo.toml --all -- --check` - `cargo check --manifest-path runner/Cargo.toml --locked --workspace` - `cargo test -p paperclip-runner-core --lib -- qualified_launch trusted_root commonjs`: 12 passed. The new tests are: - `an_artifact_under_an_immutable_trusted_root_runs_in_place_without_a_digest`: trusted-root hit, with no memfd, the same inode as the source, and no re-hash. - `an_artifact_outside_the_trusted_root_keeps_the_sealed_snapshot`: a path outside the root is sealed, and a wrong digest still fails. - `a_sibling_with_the_root_as_a_name_prefix_is_outside_the_root`: `rom2/` does not match `rom`. - `a_root_that_is_not_read_only_falls_back_to_the_sealed_snapshot`: the real probe refuses a writable root at startup. A per-artifact mount failure falls back, and a wrong digest still fails. - `the_real_mount_probe_rejects_a_writable_filesystem` - `symlink_and_parent_escapes_from_the_root_fall_back`: a directory symlink out of the root, a `..` path out of the root, and a final-component symlink (still refused). - `a_root_must_be_absolute_canonical_and_a_directory` - `without_a_trusted_root_launch_artifacts_are_sealed_and_hashed`: flag absent, behavior unchanged. - `host_assigned_executables_ignore_the_trusted_root` - `trusted_root_artifacts_execute_the_held_file_and_reopen_it_for_nested_launches`: Runner execs from the held file, and a child re-execs `$PAPERCLIP_VERIFIED_RUNTIME_EXECUTABLE`. Both report the source file as `/proc/self/exe`. - `cargo test -p paperclip-runner-core --test stdio_mcp --test process_supervisor`: 4 and 8 passed. - Ad hoc, not committed: a privileged container with a real loop-mounted squashfs. The real `fstatvfs`/`fstatfs` probe admitted a `ro` mount and refused a `ro,noexec` mount. The unit tests replace only that mount probe, because CI cannot mount filesystems. - Not verified: a run on an agent computer with the real erofs ROM, and the latency and memory savings there. The Cloud Launcher change that passes the flag is a separate follow-up. ## Risks - This change needs review from a security owner before merge. It removes the sealed copy and the per-launch digest for admitted artifacts. That relies on the host's assertion that the root is a digest-pinned immutable image, plus the descriptor checks above. - Misconfiguration fails closed. A wrong, writable, `noexec`, non-canonical or missing root only loses the speedup, and runnerd logs a warning. There is no global "trust everything" switch, and Runner never selects a root from the filesystem it finds. - Residual trust. On an admitted file, Runner no longer detects a ROM image that differs from what the launch profile expected. On the agent computer, the Launcher derives the expected digests from the same ROM. Another host that passes the flag must pin its image the same way. An attacker with root in the guest (who could write the backing block device or remount) is out of scope; such an attacker can already control runnerd. - Allowlist. Only erofs and squashfs are accepted, because they cannot be written through any mount. A read-only bind mount of a writable filesystem is refused. A ROM that changes to another filesystem type will silently fall back to the sealed path (slower, still safe). - No migration. Behavior is unchanged when the flag is absent. Older runnerd builds ignore the flag. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, container builds). Extended reasoning was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 17:46:33 -08:00
DottaandClaude Opus 5.5	f7c85bca18	test(runner-e2e): delete each runner E2E instance when it ends and expire kept instances after 6 hours (#20246 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness boots a full temporary Paperclip instance per wave: embedded Postgres, workspaces and caches, often 5–13 GB > - At wave end it deleted that instance only when the wave passed with no harness error and no `--keep` > - Almost every chaos wave has at least one failing task, so almost every instance stayed on disk > - Over Nov 6–9, 359 `paperclip-runner-e2e-` directories filled the machine's disk (ENOSPC), which stopped a running storm wave > - This pull request deletes the instance at the end of every wave, smoke and baseline run unless it is explicitly kept or its processes could not be stopped, and expires kept instances after 6 hours > - The benefit is that chaos waves can run back to back without filling the disk ## Linked Issues or Issue Description What happened?* `main.ts` kept the temporary root whenever `keep \|\| anyFailure \|\| harnessError`. A failing task is the normal output of a chaos wave, so instances accumulated until the disk was full. The codex K13 storm w-20261109T113925-s9403 died after about a minute with every task cut off. Steps to reproduce 1. Run any wave in which at least one task fails. 2. The `paperclip-runner-e2e-` directory under the OS temp dir stays behind, with no expiry. Expected behavior* The instance is removed when the wave ends. Evidence is already in `results/`. Only an explicit `--keep`, or a stop failure that may leave processes running, keeps it, and kept instances expire. Actual behavior Every failing wave leaks its instance permanently. ## What Changed - New shared `tests/runner-e2e/instance-retention.ts`: - `releaseInstance` deletes the root unless `--keep` was passed or the processes could not be stopped. In those cases it writes a `.paperclip-runner-e2e-kept.json` marker (owner, reason, time) and sets the directory to 0700. An optional `retain` list keeps small evidence files and removes everything else. - `sweepExpiredKeptInstances` removes `paperclip-runner-e2e-*` directories that carry the marker, are older than 6 hours, and have no live `postmaster.pid` anywhere inside them, and that no live process command line references. Each root is made removable first, and one failed delete does not stop the sweep. Unmarked directories are never touched, so existing directories from older runs stay until someone decides on them. `recovery_state` markers never expire. - `createRunnerE2ETemporaryRoot` runs the sweep, so every caller gets it. - Callers: - A wave whose Daytona cleanup is still pending (environment DELETE blocked, sandboxes still running after cleanup, or cleanup errors) keeps its instance as `recovery_state`, because its database owns that cleanup. - The wave releases its instance before writing the ledger and summary, so a failed report write cannot skip cleanup. - Restart chaos `main.ts` (crash-point suite included) releases the instance in its final cleanup, which also runs on failure and interrupts. The crash-point task-count check moved above supervisor creation so that error no longer leaks a fresh root. - `ttft-baseline.ts` releases after stopping the supervisor, including on SIGINT and SIGTERM. - The rolling-deployment and controller-replacement smokes now delete their roots, keeping only the evidence report, logs and screenshots. If cleanup is unconfirmed, the root is kept with a marker. - `launch.ts` marks its unconfirmed-remote-cleanup root as `recovery_state`, which never expires, and its `PAPERCLIP_RUNNER_E2E_KEEP_FAILED_PRIVATE` root as `keep`. Its existing deletion path is unchanged, and the file gets one line shorter. - README: the instance lifecycle and `--keep`. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts`: 328 files, 6915 passed. The new retention tests cover: - deleting a finished instance, and retaining only listed evidence; - keeping it with a marker and mode 0700; - sweeping only marked, expired instances whose Postgres isn't running, while sparing fresh, live, recovery-state, unmarked and unrelated directories; - a stale `postmaster.pid`; - sweeping on `createRunnerE2ETemporaryRoot`. The create-sweep test failed before `createRunnerE2ETemporaryRoot` ran the sweep. - `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 221 passed. - `node scripts/typecheck-test-projects.mjs tests/runner-e2e` and `tests/restart-chaos`: clean. - No wave or smoke was run, because the disk is still nearly full. ## Risks - A failed wave's instance is no longer available for post-mortem DB inspection unless the wave ran with `--keep`. Evidence bundles in `results/` are unchanged. - The sweep relies on `postmaster.pid`. An instance whose Postgres was started outside the harness, without that file, could be removed after 6 hours. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 09:20:09 -06:00
DottaandClaude Opus 5.5	347d21fcbd	test(runner-e2e): scope each agent's sandbox to its own run's saved state (#20253 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The runner E2E chat evals run real agents against a local Paperclip instance on a CI host > - PR #20044 hides the eval harness from local native agents with a Linux Landlock domain around each runnerd launch > - That domain is the same for every run in an attempt, so an eval agent can still read another run's saved state > - Recent chat-eval runs show agents that read other runs' OpenCode session databases, another run's run log, the instance `pg_wal`, `control-plane-state.json`, and `/proc/<pid>/cmdline` of harness processes > - This pull request scopes each runnerd launch to its own run and moves the selected case id off the command line > - The benefit is that an eval agent cannot learn about the case or other runs from files outside its own run ## Linked Issues or Issue Description Refs #20044 What happened In local native OpenCode chat-eval cells, worker agents read files that belong to other runs or to the harness: - other runs' OpenCode sqlite session databases under `runtime/paperclip-runner/durable-sessions/` (a status-cards run) - `/proc/<pid>/cmdline` of harness processes, which shows harness paths and the case id (`routines.pause-owned`) - another run's run-log ndjson, and a listing of the instance `db/pg_wal` (`routines.pause-owned`) - a listing of `control-plane-state.json` (`runs-adapters.list-model-options`) The Landlock sandbox from #20044 hides only the harness (`tests/`, the Git store, private attempt directories). All runnerd launches share one hidden set, so the instance's per-run directories stay readable. Expected behavior An agent process sees only its own run's durable session, OpenCode session and provider trace. Other runs' sessions, run logs and traces, the instance database files, secrets and the shared control-plane state in the attempt instance are denied. The case id is not on any harness command line. Steps to reproduce 1. Dispatch a local native OpenCode chat-features cell that has two agents (for example a status-cards case). 2. In the worker's shell, run `ls <instance>/runtime/paperclip-runner/durable-sessions/*/` and `ls <instance>/db/pg_wal`. 3. Run `cat /proc/<launcher pid>/cmdline`. 4. Before this change, all three succeed and show other runs' files and `--id chat-features...`. ## What Changed - `agent-filesystem-sandbox.py`: new `--expose PATH` option. It grants a path beneath a hidden directory. Landlock does not check traversal, so the run uses its own entry while the parent stays unlistable. The helper refuses a missing exposure, an exposure outside every hidden path, or one that would reveal a hidden path. It fails the launch unless every exposed path is readable. - `agent-filesystem-sandbox.ts`: the launcher passes the instance root in the sandbox configuration. `runnerdLaunchSandbox` adds per-launch hidden paths: `durable-sessions/`, `runner/`, `control-plane/`, `opencode/` under `runtime/paperclip-runner/`, and `data/run-logs/`, `data/provider-traces/`, `data/backups/`, `db/`, `secrets/`. It exposes only the launch's own durable-session directory (from `--state-dir`), Codex runner state directory, OpenCode session directory (from `--session-id`, named as the OpenCode driver names it) and provider trace file. - The wrapper creates the exposed entries, and the shared instance directories that the product creates on first use, before runnerd starts. It logs the run-scoped hidden and exposed paths for each launch. - New `probeRunScopedSandbox`: at server start, a sandboxed shell runs against a fixture instance. Another run's session, OpenCode database, runner state, run log, trace, `pg_wal`, secrets, control-plane state and the server's `/proc/<pid>/environ` must be denied. The run's own session, OpenCode directory, ACPX directory and trace must be writable. In CI any failure stops the attempt. The result goes to `server.log`. - `selectors.ts` and `launch.ts`: the launcher reads the selected cell from `PAPERCLIP_RUNNER_E2E_SELECTED_ID` and removes it from its environment. - `runner-full-stack-e2e.yml` (Run paid cell) and `chat-catalog-testops-case.yml`: pass the cell in that variable instead of `--id`. A target branch that does not read the variable still gets `--id`. - `SECURITY.md`: documents the run scope, the `/proc` limit and what stays readable. ## Verification - `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/agent-filesystem-sandbox.test.ts tests/runner-e2e/workflow-security.test.ts tests/runner-e2e/manual-private-workflow.test.ts tests/runner-e2e/private-trust-workflow-security.test.ts` passes on macOS. The Landlock host tests (including the new fixture-instance probe test) run only on Linux CI. - `npx tsc --noEmit -p tests/runner-e2e` shows no errors in the changed files. - Paid run [42611358428](https://github.com/paperclipai/paperclip/actions/runs/42611358428): `chat-features.runner-opencode.local.agents.change-manager` on the same commit applied to `chat-eval/master-candidate` (`fca4bb7ce3`). Result: passed. In `server.log` the setup probe reports `failures: []` and `run:otherDurableSession`, `run:durableSessionListing`, `run:otherRunnerState`, `run:controlPlaneState`, `run:otherOpencodeSession`, `run:otherRunLog`, `run:databaseWal`, `run:secrets`, `run:otherProviderTrace` and `run:harnessEnviron` all `denied`; `run:ownDurableSession`, `run:ownOpencodeSession`, `run:acpxRuntime` and `run:ownProviderTrace` `allowed`; `run:harnessCmdline` `allowed` (recorded, Landlock limit). The runnerd launch line lists the nine run-scoped hidden instance paths and exposes only its own durable-session and OpenCode session directories. The agent finished the case inside that sandbox. ## Risks - A sandboxed runnerd cannot create new direct entries in the instance root, `runtime/`, `runtime/paperclip-runner/` or `data/`. The wrapper creates the known ones first (`acpx`, `companies`, `projects`, `workspaces`, `runtime-context-assets`, `skills`, `runtime/provider-packs`). A provider that creates a different new directory there fails in a sandboxed local cell. The paid run covers OpenCode only. ACPX cells keep their runtime directory readable. - A run still reads its own session's control-plane state, and other agents' homes and task workspaces in the same instance. - Landlock cannot hide `/proc/<pid>/cmdline`. Harness paths stay visible there. Only the case id moves to the environment. Scheduled multi-cell campaigns still pass `--id`. - Harness-only change. No product code changes. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), extended thinking, tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-09 07:28:12 -06:00

1 2 3 4 5 ...