PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-09 16:02:36 +01:00

Author	SHA1	Message	Date
DottaandClaude Opus 5.5	a35ec8b845	fix(native): hand a shutdown-interrupted startup to the next controller Graceful shutdown closes the runtime API listener before it drains heartbeat runs. A native execution between provider dispatch and runnerd backend setup then failed with local_runtime_api_listener_unavailable, so the drain no longer saw it as running and never suspended it for the next boot. resolveLocalRuntimeApiUrl now raises LocalRuntimeApiListenerClosedError (same failure code) once this process stopped serving, and the executor treats it as a restart detach: the run stays running with its claim, the drain suspends it, and the next controller adopts it. A listener that never started is still a startup failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 13:25:02 -06:00
Dotta	65aee998a3	fix(server): settle empty environment cleanup notifications (#19316 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Environment workers retain cleanup duties and retry them safely. > - A metadata change can notify the worker without adding cleanup debt. > - The pending-cleanup worker kept a work hold during its thirty-second spacing window even when its owner reported no debt. > - Historical cleanup already checks that condition without advancing a provider sweep. > - This PR applies the same owner check to pending cleanup and preserves later notifications. > - An empty notification can settle while real cleanup and uncertain work still block sleep. ## Linked Issues or Issue Description Refs #18646. That change covers empty historical-cleanup notifications. This follow-up covers the pending-cleanup queue. Related runtime work: #18988. A search of open and closed PRs and issues found no duplicate pending-queue fix. This repairs an existing worker under the roadmap's Cloud deployment work. What happened? A normal metadata update on a released lease left one tracked work intent and a scheduled pass. The actual cleanup deadline was null before and after the update. The worker kept the hold until its spacing window ended. Expected behavior The worker can clear an empty notification through its existing owner predicate after a normal sweep and deadline read complete. It must retain new debt, unresolved transactions, failed sweeps, failed reads and later notifications. Provider sweep spacing and cleanup retry deadlines must stay unchanged. Steps to reproduce 1. Create and release a fixture cleanup lease through the normal services. 2. Start the registered worker and confirm that its cleanup deadline is null. 3. Change the released lease's metadata before the thirty-second spacing window ends. 4. Run the new test. The original worker keeps an active intent even though eligibility remains empty. Paperclip version or commit Candidate: `69aaee95cd73322a37e0270354d4a06bf5512ba7`. It normally merges actual master `cd3832d2357eab481a861331ba1bc944a0b951f4`. The focused retry repair follows the original two-file fix. Every other tree blob still matches that master. The diff has two files. Deployment mode Source checkout with external PostgreSQL 17 and controlled coordinator time. No deployed host or database suspension result is claimed. ## What Changed - Apply the existing cheap deadline check to both registered cleanup queues during provider spacing. - Clear only the captured notification generation when the owner returns no deadline and the latest normal sweep completed successfully. - Invalidate that coverage before every full sweep. Startup and later sweep failures keep their existing retry even if a metadata notification has no database deadline. - Add real PostgreSQL controls for empty metadata changes, concurrent debt, unresolved transactions, failed reads through drain/resume, and unread-spool recovery after startup and later sweep failures. - Retain the original provider sweep, retry, deferred-buffer, creation, startup, historical and shutdown controls. ## Verification Real PostgreSQL red/green qualification passed. The original empty-notification counterfactual failed at active work one, expected zero, after actual null eligibility reads (16.706s). Review then found a missing failed-sweep combination. Two additional cases using the real malformed spool and ordinary metadata notifications reproduced lost retries on the earlier published head, both at the intended pending predicate (42.144s). One case fails at startup; the other first completes a successful sweep, then fails a later sweep. Failed fixtures and original logs remain retained. At measured source `1ccd227c51e03ab4306d7cb85cf81efb0e3d3678`, all 29 owner cases passed in 113.061s, without filters or skips. The tests repair the actual spool, recover the ordinary cleanup lease, perform provider cleanup once, preserve the later-sweep backoff, and return to quiet. Normal migrations used a fresh owned external database; each pool was capped at four with one worker. The first test-fixture attempt accidentally replaced the real spool driver; that setup failure is preserved separately and was corrected before the counterfactual. No final cleanup result or receipt was seeded. Run the full owner suite against a fresh, owned external test database with normal migrations applied: ```sh pnpm --filter @paperclipai/server exec vitest run src/__tests__/environment-cleanup-work.test.ts --maxWorkers=1 --no-file-parallelism ``` Set `PAPERCLIP_ENVIRONMENT_CLEANUP_TEST_DATABASE_URL` to that fixture. Local qualification uses one worker and a pool of four. Current native TypeScript checks passed: normal production declaration layers (44.127s) and the affected test entry using the repository's actual CI options and references (3.922s). Own dependency builds passed. A frozen install first found stale upstream ACPX patch metadata; normally carrying #19294 corrected the hash references for the same package version. No lock policy was relaxed. Full local workspace typecheck, tests and build were not run in this bounded external-PG lease. Fresh checks on the current head `69aaee95cd73322a37e0270354d4a06bf5512ba7` completed with 53 successes, one neutral Pony review and two skips, with no pending or failed gates. Its [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/42104787089), aggregate verification and commitperclip review succeeded. The complete latest edited Greptile and Pony summaries both gave 5/5. Both review threads are resolved, and all root and nested thread pages were inspected. The current head only removes four changed code-comment lines to follow CONTRIBUTING.md. Its emitted JavaScript is byte-identical to the measured parent `1ccd227c51e03ab4306d7cb85cf81efb0e3d3678`; tests, configuration and dependencies are unchanged. The local 29-case evidence therefore carries to this head without a duplicate local run. The remote results above are fresh for the current head. On the earlier head, Runner CI failed an omitted-context checkpoint-settlement case. The fresh Runner job on this head passed without a rerun. The original failure logs are preserved and its cause remains unproven; current success is recovery evidence, not a causal explanation. The recorded test and migration command PIDs were reaped; an independent audit verifies zero owned database clients before normal cleanup. Failed fixtures remain retained. The process audit covers recorded command PIDs, not an exhaustive history of child processes. The long quiet assertion uses controlled coordinator time and actual SQL counters. It does not prove a real twenty-minute quiet window, provider behavior or deployed suspension. ## Risks The empty read must not erase a notification that arrives while it runs. The captured generation and transaction controls preserve that boundary. An unknown or failed read keeps work pending. Empty deadline rows cannot cover a failed sweep or an unread local spool; only a successful normal sweep and deadline read establish that coverage. No cleanup row, resource claim, provider action, authorization rule, sweep spacing or retry deadline is removed. ## Model Used OpenAI `gpt-6.1-sol` produced the delegated regression tests and source review. OpenAI Codex, based on GPT-6, implemented the small worker change and managed integration and verification. Both used repository tools and code execution. Exact served backend revisions and context limits were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked an existing public issue or described the issue using the issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal issue ID - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have checked that no documentation change is needed for this internal worker fix - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all reviewer comments before requesting merge	2026-11-06 13:04:40 -06:00
DottaandPaperclip	f513f0efd9	fix(recovery): admit certified process-loss preparation retries (#19552 ) ## Thinking Path > - Paperclip keeps task work durable across server restarts. > - Restart recovery records a certificate when a Runner preparation ended before provider work started. > - Automatic recovery already verifies that certificate together with saved files and stopped leases. > - Explicit Retry called a preparation helper that only reached this proof for status `interrupted`. > - A certified `failed/process_lost` source therefore stayed blocked after its files were recovered. > - This pull request makes the helper reuse the same existing proof. > - A queued user Retry can then admit one new turn while all current safety and policy gates remain effective. ## Linked Issues or Issue Description What happened? A task had a certified process-lost preparation, verified saved-workspace receipts, a destroyed source sandbox, and one or two later attempts refused before dispatch. Its accepted Retry request stayed deferred with `execution_recovery` and created no successor. The explicit admission helper excluded this source before it could use the preparation proof added in #19464. This also prevented the refused-attempt continuation path from #19466 from advancing. Expected behavior Explicit Retry must use the same positive pre-provider proof as restart recovery. It must preserve source history, refuse contradictory execution evidence, honor current policy gates, and admit at most one successor for the existing request. Steps to reproduce Persist a legacy preparing run as failed with `process_lost` and the host bootstrap certificate that provider work did not start. Record its canonical committed snapshot, recovery/finalization operations, clean workspace authority, and exact destroyed-lease receipt. Add one or two unstarted cancelled descendants with the dispatcher's execution-wait receipt. Accept one Retry request and resume it concurrently twice. Before this fix, no successor exists and the same request remains deferred. Paperclip version or commit Reproduced on master `a6c22806308e8a3c7b21145f80fb471756bb443d`. Related repairs: #19464 and #19466. ## What Changed - Reuse `isInterruptedNativePreparation` before the existing failed-preparation checks. The existing no-coordinator and historical-ancestor restrictions remain. - Add real-database regressions for one and two refused descendants, duplicate clicks, concurrent resume, continuation dispatch, preserved history, and unchanged recovery attempt counts. - Add late-gate checks for provider work, provider events, a newer native checkpoint, missing bootstrap proof, missing stop proof, a changed snapshot, formal approval, user Stop, agent/company pause, and a spent budget. - Clarify that explicit admission uses the existing certified preparation proof. ## Verification - Red: both real-database cases passed the saved-file, interrupted-preparation, and refused-dispatch proof checks, but the original successor-count assertion found zero queued runs instead of one. The accepted requests retained `execution_recovery`. - Green: the same cases admit one successor after two concurrent resume calls. Continuation dispatch resolves to the latest refused attempt. Source and cancelled-attempt history stay intact. - Complete relevant local suites: 456/456 passed before rebase. Command from `server`: `../node_modules/.bin/vitest run --config vitest.config.ts --fsModuleCache src/services/task-retry.test.ts src/services/explicit-native-continuation.test.ts`. - Final head `9788382c019df57899af06e81211e7c5165e7f0d`: the thirteen new cases passed 13/13 again after rebase. Required CI and review remain pending. - Environment: macOS ARM64, Node 26, Vitest 5.0.3, disposable real PostgreSQL. Tests exercise production request, admission, and continuation services. No provider executes and no hosted task is changed. - Hosted recovery still requires rollout and verification through the original queued requests. Final reviewed head: `9788382c019df57899af06e81211e7c5165e7f0d`. [Required CI](https://github.com/paperclipai/paperclip/actions/runs/42273663624) passed (56 successful checks, four expected skips, none pending or failed). Greptile and Paperclip Review both rated this head 5/5 with no findings; all review threads are resolved. ## Risks The change accepts only the existing verified preparation contract. An error label alone is insufficient. Provider-started or contradictory state remains blocked. Saved-file, lease, approval, Stop, pause, and budget checks remain effective. No schema, retry limit, provider setting, account selection, or production scheduler changes. ## Model Used OpenAI GPT-6 family via Codex, with coding and tool use. The exact model ID and context window are unavailable in this session's metadata. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 13:03:21 -06:00
DottaandClaude Opus 5.5	71313207eb	fix(runner): continue after a restore-lost turn the checkpoint never bound (#19539 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native Codex and ACPX runs survive control-plane restarts: the runner restores the provider conversation, and the control plane continues the task from its checkpoint > - When the runner restores a conversation after process loss, it reports the turn that was active as lost on restore (`provider_turn_lost_on_restore`) > - A controller can die after the provider accepted a turn but before that turn reached the checkpoint. Then the next recovery does not know the turn that the runner reports as lost > - In a restart chaos wave with two hard kills about 20 s apart, this made the second recovery fail with "Provider turn terminal did not name the active turn". The session close then became unrecoverable > - This pull request lets recovery continue: the transport no longer stalls a new turn start behind that report, and the Codex session records the report as restored history > - The benefit is that a task continues automatically after back-to-back restarts, instead of failing and needing an operator ## Linked Issues or Issue Description What happened? A Codex task was resumed after a hard restart. A second hard restart came about 20 s later. The second recovery failed at once with `NativeProviderTerminalFailure: Provider turn terminal did not name the active turn.`, and automatic recovery stopped. The session close could not settle, which quarantined later work (that blast radius is fixed separately in #19532). Steps to reproduce 1. Start a native Codex task. Kill the control plane (SIGKILL) shortly after the provider accepts a turn and before the next checkpoint is saved. 2. Let restart recovery resume the run. Kill the control plane again while the resumed run is active. 3. Observe the second recovery. Durable evidence from the wave (runner journal of the run): - Restart 1 (17:32:34.568): the provider accepted continuation turn T2. The controller was killed 170 ms later, before T2 reached the checkpoint. - Recovery 1 started a new turn T3. The runner command result named T3 (seq 15). The restored runner then reported T2 as lost on restore (seq 20) before `turn.started` for T3 (seq 23). The transport held the T2 terminal while the T3 start was pending, and stopped pumping at it. So T3 ran in the provider (items, a tool call) but the control plane never accepted or checkpointed it. - Restart 2: recovery found no active turn. The restored runner reported T3 as lost on restore. The Codex session had never bound T3, so it failed the protocol (`turn_binding_mismatch`). Expected behavior Recovery continues the task after each restart (R1). Actual behavior The run failed with a protocol error. Its close could not prove settlement, so cleanup needed an operator. ## What Changed - `runnerd-codex-transport.ts`: while a turn start is pending, the event pump holds a terminal only when the terminal can belong to that turn. When the durable command result names the new provider turn and the terminal names a different turn, the pump delivers it and continues to the new `turn.started`. A terminal that names no turn, or any terminal before the command result, is still held. - `codex-session-notifications.ts`: a `turn/completed` with status `failed` and a recoverable `provider_turn_lost_on_restore` error, for a turn that this session never bound (not active and not terminal), is recorded as restored history with a `harness.diagnostic` (`unbound_restored_turn_lost`). The session keeps its own turn. The runner reports this cause only after it proved that the old provider process ended and restored the conversation, so that turn can produce no further work or result. - Other terminals for an unbound turn (a completion, an ordinary failure) still fail with `turn_binding_mismatch`. ## Verification - Red before the fix, real `paperclip-runnerd` and fake Codex app server: `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/live/runnerd-restart-continuation.test.ts -t "never bound"` failed with `NativeProviderTerminalFailure: Provider turn terminal did not name the active turn.` at `codex-harness-session.ts:207` via `native-session-runtime.ts` (the same stack as the wave). - Green after the fix: that test completes the run on `provider-turn-2` with no `session.failed` and no `turn.failed`. - New driver tests in `codex-app-server-driver.terminal-replay.test.ts`: the restore-lost unbound turn is recorded (also on replay) and the session's own turn completes. A completion or ordinary failure for an unbound turn still fails the protocol. - New transport unit test for the hold rule (`does not hold a pending turn start behind another turn's restore-lost terminal`). - Existing suites pass: `src/drivers/codex/`, `src/native-session-runtime.test.ts`, `src/live/runnerd-restart-continuation.test.ts`, `src/live/runnerd-controller-outage.test.ts`, `src/live/runnerd-codex-transport-warm-attachment.test.ts`, `src/live/runnerd-codex-transport.test.ts` (turn-start cases). Three tests timed out once at host load 51 and passed when rerun alone. - `pnpm --filter @paperclipai/paperclip-runner typecheck:typescript` passes. - Live replay of the wave task: results are added in a comment below. ## Risks - The transport change applies only when the command result already named the new turn and the terminal names a different turn. A terminal for the new turn keeps the existing fence that puts `turn.accepted` first. - The session change accepts only the runner's restart-interruption cause. A lost turn's work is already in the restored conversation. The continuation turn tells the agent to reconcile unfinished work. - No profile, schema or pack contract changes. ACPX sessions that use the same Codex harness session get the same behavior. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 13:02:25 -06:00
DottaandClaude Opus 5.5	c6eecac1e0	fix(heartbeat): keep a shutdown-suspended native recovery for the next boot (#19530 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner Full-Stack E2E workflow tests the Runner in Daytona sandboxes from a signed, content-addressed image > - Every change to an audited Runner input needs a new image before a Daytona cell can test it > - The image job took 10 to 16.5 minutes today, and a restart chaos campaign waits on it after each runner-side fix > - Measurement shows two serial waits: a full Rust dependency compile (about 5 of 6.6 minutes) and a 3.5 to 4 minute download of cached base layers that starts only after the builder stages finish > - There is also no way to build only the image: every dispatch reaches a paid gate that the operator must reject > - This pull request removes both serial waits and adds an `image_only` dispatch input > - The benefit is a shorter fix-to-Daytona-test loop, with every image security check unchanged ## Linked Issues or Issue Description What happened? The `Publish verified Daytona image` job is on the critical path of the control-plane restart chaos campaign. Builds today took 16.5 min (run 42240434011), 15 min (42253920727), 10.5 min (42259118276) and 16 min (42265637650). Run 42264611573 failed: its RunsOn runner lost communication with GitHub 7.6 minutes into the build step, during the Rust compile. GitHub has no log for that job. Expected behavior A Runner source change should not recompile the locked third-party Rust graph. Cached image layers should download while the builder stages compile. An operator should be able to publish only the image, without a paid gate. Steps to reproduce Read the BuildKit timeline in the logs of job 127970125495 (run 42240434011). See the stage table below. ## What Changed - `docker/daytona-runner/Dockerfile`: compile the locked Rust dependency graph in its own layer against placeholder targets. The layer depends only on `Cargo.toml`, `Cargo.lock` and the crate manifest. The step deletes the placeholder outputs and fingerprints, so the next step always compiles the real Runner sources. The final stage still receives only the stripped executable. - `docker/daytona-runner/Dockerfile`: move the content-ID / source-revision check above the two `COPY --from` steps. It does not depend on either builder stage, so BuildKit now downloads the cached base and CLI layers in parallel with the compile. Both copies use `COPY --link`. - `.github/workflows/runner-full-stack-e2e.yml`: add the boolean `image_only` dispatch input. `authorize`, `target_lock`, `catalog` and `daytona_image` run unchanged. Every other campaign job gets `&& inputs.image_only != true`. `authorize` rejects `image_only` together with TestOps inputs. - `tests/runner-e2e/workflow-image-only.test.ts` (new): pin the job classification, the top-level guard on each campaign job, the guard on every protected environment, and the TestOps rejection. - `tests/runner-e2e/README.md`: document the layer layout and the `image_only` command. ## Verification Measured on the same RunsOn `m7i-flex.xlarge` (4 vCPU, 16 GiB, native x64) runner class. "Before" is master job 127970125495 (run 42240434011). "After (cold)" is this branch, job 128077687214 (run 42271236140), with no dependency layer in the registry cache yet. Its paid gate was rejected. \| Stage \| Before \| After (cold dependency cache) \| \|---\|---\|---\| \| Whole image job \| 16.5 min \| 11.6 min \| \| Rust dependency graph \| 311 s (inside one 399 s step) \| 395 s (own layer; reused from cache after a master build) \| \| `paperclip-runner-core` compile \| ~87 s \| 65 s \| \| Cached base and CLI layer download \| 236 s, after cargo finished \| 257 s, in parallel with cargo \| \| Copy runnerd + provider pack \| 20 s + 50 s \| 9 s (`--link`) \| \| Provider pack stage \| 143 s (parallel) \| 152 s (parallel) \| \| Export + push \| 82 s + 10 s \| 81 s + 11 s \| \| Dependency layer size \| n/a \| `target` 852 MB + registry 333 MB (builder stage only) \| With a warm dependency cache, a Runner source change compiles only the 65 s crate step. The critical path then becomes the ~4 min cached-layer download or the provider pack. The expected job time is about 7 to 8 min. I measure this after merge with an `image_only` dispatch (master first, to write the cache, then a Rust-only branch change). Disk: the instance starts with 38.76 GiB free. The dependency layer adds about 1.2 GB uncompressed to builder storage. The final image does not change. - `vitest run tests/runner-e2e/workflow-image-only.test.ts tests/runner-e2e/workflow-security.test.ts tests/runner-e2e/daytona-image.test.ts`: the new test and all image tests pass. Three tests in `workflow-security.test.ts` (`gates every provider-secret job with stable actor IDs`, `keeps provider credentials inside explicitly gated paid workflows`, `isolates private archive trust from workflows that execute target-branch code`) fail on `origin/master` without this change, because of `chat-catalog-testops-case.yml`. This pull request does not change them. - `node --test tests/runner-e2e/product-regression-testops.test.mjs`: 15/15 pass. - No local Docker build. The Dockerfile is measured with a branch-targeted CI dispatch whose paid gate was rejected. ## Risks - The dependency layer keeps the Rust `target` and registry trees in a builder-stage layer, and default-branch builds export it to the registry cache. This adds builder disk and cache upload when `Cargo.lock` changes. The build prints the size (`du -sk target /usr/local/cargo/registry`). - The dependency cache helps only after a default-branch build writes it. Branch builds read it but never write it. This is the existing cache policy. - `image_only` changes the workflow, so it works only after merge to master. - No check is removed. Lockfile verification, cosign sign and verify, the anonymous pull check, the provider-pack manifest match, the default-branch-only cache write and the actor allowlist are unchanged. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, GitHub CLI). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 13:01:16 -06:00
DottaandClaude Opus 5.5	7afb57c022	test(runner-e2e): repair restart-cell harness drift (#19538 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner Product E2E suite proves that agent work survives a control-plane restart > - A restart chaos campaign found three restart cells that fail in the test harness, not in the product > - A harness failure hides the real restart result, so the campaign cannot classify these cells > - This pull request repairs the harness drift in those cells and names the one failure the harness cannot prevent > - The benefit is that these cells now test restart behavior, or fail with a clear cause ## Linked Issues or Issue Description Refs #14811 (connector setup became one screen). Refs #19475 (completion-update restart fixture). What happened? 1. `lifecycle-baseline.runner-codex.local.tool-review-restart` clicked a "Save and continue" button. #14811 removed that button. The cell failed during setup. After that fix, the cell also failed after the restart: the expanded review card does not survive the post-restart reload, so "Approve & run" was not visible. 2. `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` failed with `page.goto: net::ERR_ABORTED` after the controller restart. The old document's Vite client reloads when the replacement server starts, and this aborts the test navigation. The code tolerated only `TimeoutError`. 3. `completion-updates.runner-acpx-claude.local.handoff-completion-restart` waited for the source agent to run an instructed wait command. Claude replied without the command (2/2 attempts), so the restart never occurred. The cell then failed with a generic timeout. Expected behavior Each cell reaches its restart boundary and tests restart behavior. If the boundary cannot be reached, the cell names the cause. Steps to reproduce Run `pnpm test:e2e:runner -- --max-parallel 1 --max-automatic-retries 0 --id <cell>` for each cell above on `master`. ## What Changed - `tests/runner-e2e/connection-reviews.ts`: accept the one-screen "Check link" step and the older "Save and continue" step. This uses the same regex approach as `connection-flow.ts`. - `tests/runner-e2e/runner.spec.ts` (tool review): click "Review request" after the restart reload, on the document that acts on the card. - `tests/runner-e2e/chat-restart.ts`: add `gotoAfterServerRestart`. It retries only `net::ERR_ABORTED`, within the caller's deadline and at most 5 attempts. It still accepts `TimeoutError`. It rethrows all other errors. The structured-question restart uses it. The existing sentinel and durable-state assertions still prove the new document. - `tests/runner-e2e/completion-updates.ts` and `completion-update-flow.ts`: add `completionRestartGateSkipped` and `completionRestartGateTimeoutDetail`. The gate poll stops at once with `restart boundary not reached (agent skipped gate)` when no completion turn is active and the turn that carries this task's Done facts (`chatCompletionUpdates`, as in `completionDelivery`) succeeded or published its reply before the gate reports `waiting`. A failed or cancelled turn is not blamed on the agent, because delivery can retry it; the timeout detail names it instead. The post-restart chat navigation now uses `gotoAfterServerRestart`. - A fixture-level gate is not possible without a product hook. The native run finalizer publishes the completion reply inside its own transaction, through the issue service closure. The isolated-server entrypoint can patch only class prototypes and HTTP responses, and the reply uses no tool call or HTTP request. - `tests/runner-e2e/README.md`: document the skipped-gate failure. - Docs: use `pnpm --filter @paperclipai/paperclip-runner run build:runner-binaries` in `tests/runner-e2e/README.md`, `tests/restart-chaos/README.md`, `tests/restart-chaos/main.ts` and `doc/DEVELOPING.md`. - No product code changes. ## Verification - Unit: `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-restart.test.ts tests/runner-e2e/completion-updates.test.ts tests/runner-e2e/connection-reviews.test.ts` (36 passed). - Typecheck: `node scripts/typecheck-test-projects.mjs tests/runner-e2e` passed. - Live cells, local, `--max-parallel 1 --max-automatic-retries 0`, macOS, load average 35–65: - `lifecycle-baseline.runner-codex.local.tool-review-restart`: PASS (first run found the review-card reload drift, fixed in the second commit). - `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume`: PASS. A first attempt failed before the restart with `PRP command session.open timed out` at load ~36 (environment). - `completion-updates.runner-acpx-claude.local.handoff-completion-restart`: FAIL as designed, twice (before and after the review fix), with `restart boundary not reached (agent skipped gate): completion turn … ended succeeded without the reference gate reporting waiting`. Claude skipped the wait both times, so this cell still does not exercise a restart. ## Risks - Low risk. Test harness and docs only. - The navigation retry could hide a real abort. It is limited to `net::ERR_ABORTED`, 5 attempts and the deadline, and the test still asserts that a new document loaded. - The ACPX Claude completion restart cell still does not exercise a restart when the agent skips the gate. It now reports this cause by name. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edit, browser test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:59:54 -06:00
Devin FoleyandPaperclip	fd4b197f83	fix(ui): hide runner selection during agent setup (#19517 ) ## Thinking Path > - Paperclip lets people manage AI agents for work. > - Agent setup connects a harness, model, account, and computer. > - The system already selects a qualified runner for a new agent. > - Setup still asks users to select that internal runner. > - The runner dropdown can be the only field in Advanced. > - This PR removes the dropdown and hides an empty Advanced section. > - Users keep their saved execution settings and see fewer setup controls. ## Linked Issues or Issue Description What existing behavior does this improve? Agent creation, onboarding, built-in agent setup, company import, and agent configuration. Subsystem affected The UI, setup error messages, tests, and Storybook fixtures. Current behavior Users can select a runner from a dropdown. Some setup screens place only that dropdown in Advanced. Errors can direct users to that control. Proposed behavior Remove runner dropdowns from setup and configuration. Use the existing automatic selection for new agents. Retain saved runner settings. Show Advanced only when other fields remain. Reason and benefit Users can configure agents without an extra internal execution choice. Setup also avoids an empty Advanced section. Breaking changes Users can no longer select a runner through these UI dropdowns. The API and CLI retain their runner options. Existing agents keep their saved execution settings. Additional context Related: [Default new agents to the qualified runner for their harness](https://github.com/paperclipai/paperclip/pull/15422). This PR changes the existing setup UI. It adds no harness or runtime default. The master merge retains the custom gateway placement from [#19510](https://github.com/paperclipai/paperclip/pull/19510). ## What Changed - Remove the shared runner dropdown and its setup/configuration callers. - Hide Advanced when it has no remaining fields. Keep command and argument settings where supported. Preserve custom gateway setup under Advanced on the Claude and OpenAI API key paths. - Keep setup failures visible and keep the retry action outside the removed control. - Update errors and documentation that referred to the removed dropdown. - Update component tests and Storybook fixtures for the new UI states. - Make Legacy first-task evals resume an API-configured setup record. Record that path separately from fresh automatic onboarding. ## Verification - Latest master integration (`187c0f6d6e5`): 276 onboarding/provider-connection tests passed across six files. UI typecheck and build passed. Token gates, query-error rendering checks, and the Storybook catalog check passed. The conflict resolution retains master's bounded connection tests and cancellation while removing the obsolete runner instruction. The first test command used the wrong working directory and failed before collecting tests; the corrected invocation from `ui/` passed. - Earlier branch verification passed: full workspace build and typecheck, 1,037 focused UI/service tests, 49 heartbeat provider contract tests, eval typecheck, and 30 map/catalog tests. - The full local `pnpm test:run` run passed all six chat integration shards (1,931 tests). It was stopped for the master merge before the complete suite finished. The [complete CI run](https://github.com/paperclipai/paperclip/actions/runs/42264662642) passed on the previous head `72e1699`. - Current head `a097c8369d079c839e78661a2206cfa62c5210e0`: [full CI](https://github.com/paperclipai/paperclip/actions/runs/42273168268) passed. PR checks total 55 passed and 3 skipped, with none failed or pending. Greptile reviewed this head at 5/5 with no actionable findings or unresolved review threads. - Live browser/provider evals have not run. The Legacy eval change has no live qualification claim. ## Risks - Removing the dropdown changes the setup UI. Existing API and CLI runner choices remain available. - Conditional Advanced sections must retain valid command and argument fields. - Legacy first-task evals now prove setup resumption. They do not prove the fresh-agent automatic default. - No schema change or data migration is included. ## Model Used OpenAI `gpt-6.1-sol`, through Codex in T3 Code. Reasoning effort: `xhigh`. The model used file editing, shell commands, and test tools. The harness does not expose the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-06 10:58:42 -08:00
Devin FoleyandPaperclip	f0006c7783	fix(workspaces): contain inventory deadline aborts (#19543 ) A capture deadline can expire while file identity validation awaits the filesystem. Recheck cancellation before creating the inventory read stream so the expired capture rejects without terminating the server. All 224 affected capture and archive tests pass, including real Node subprocess cancellation regressions. Final-head CI passes 54 checks, and Greptile scores 5/5 with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-06 10:45:47 -08:00
DottaandClaude Opus 5.5	187c0f6d6e	fix(runner-e2e): prepare the artifact oracle for every suite that runs it (#19518 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Runner Full-Stack E2E workflow runs paid Daytona restart cells for the control-plane restart chaos campaign > - Manual run 42243751291 failed `daytona-restart-continuity.runner-codex.daytona.recover-controller` before task creation: `docker version` timed out after 10 seconds in the artifact oracle preflight > - The everyday flow requires the Python artifact oracle by case alone (`requiresEverydayArtifactOracle(caseId)`). The workflow step that starts Docker and pulls the oracle image also filters by suite, so suites that reuse `recover-controller` skip it > - On a cold RunsOn host the first `docker version` takes about 20 seconds (the everyday cell in the same run shows this), so the 10-second preflight fails when the workflow step did not run first > - This pull request gates the oracle step by case only, which matches the runtime requirement > - The benefit is that the Daytona restart cell reaches task creation instead of failing in setup ## Linked Issues or Issue Description What happened? Run https://github.com/paperclipai/paperclip/actions/runs/42243751291 job `daytona-restart-continuity.runner-codex.daytona.recover-controller` failed with `Artifact sandbox qualification failed before task creation: ... docker version ... timed out after 10 seconds`. The job log shows no "Prepare pinned Python artifact oracle image" step. Expected behavior Every cell whose flow runs the artifact oracle prepares Docker and the pinned oracle image before paid execution. Steps to reproduce Enumerate `runnerMatrix` cells where `requiresEverydayArtifactOracle(task.id)` is true. The cells in `daytona-restart-continuity`, `rolling-web-deployment`, `rolling-web-deployment-warm` and `remote-workspace-storage` do not match the old workflow condition. ## What Changed - `.github/workflows/runner-full-stack-e2e.yml`: the "Prepare pinned Python artifact oracle image" step is now gated by `matrix.caseId` only. The suite filter is removed. - `tests/runner-e2e/codex-ci-sandbox.test.ts`: the oracle gate test now checks every catalog cell, not only `everyday-workflows` cells, and rejects a suite filter. - `tests/runner-e2e/workflow-security.test.ts`: the pinned condition string follows the new gate. ## Verification - Red: with only the test change, `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/codex-ci-sandbox.test.ts` fails on `prepares the artifact oracle for exactly the stories that execute downloads`. - Green: with the workflow change, the same command passes (13/13). - `tests/runner-e2e/workflow-security.test.ts` locally shows the same 5 failures before and after this change (`pr-trusted.yml` step counts and `chat-catalog-testops-case.yml` credential checks). They do not involve this step. - Live proof: a narrow manual dispatch after merge. ## Risks - Low risk. Cells in four more suites now run a Docker version check, a pinned image pull and an inspect before paid execution. This adds about 20 to 30 seconds per cell. The image is pinned by digest. The step has no secrets. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, GitHub CLI, file edit). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:40:12 -06:00
DottaandClaude Opus 5.5	d4d3383c97	fix(runner): scope cleanup quarantine to the session's task and conversation (#19532 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs (Codex, ACPX) execute through the runner's `executeNativeSession`, which gates each new session behind cleanup of earlier sessions > - When a session close fails in a way that needs operator recovery (`NativeSessionCloseUnrecoverableError`), the runner keeps a quarantine entry for it > - The admission gate keyed that quarantine by `[companyId, backend kind, backend name]`, so one bad close blocked every later run of the same backend in the whole company > - In a restart chaos wave (two hard kills about 20 s apart), one Codex session's close became unrecoverable. After that, every Codex run and every Retry in the company failed in about 1 s with `native_session_cleanup_quarantined` > - This pull request scopes the gate to the resources that a retained owner can still hold: its task (issue) and its provider conversation (normalized session) > - The benefit is that one failed close no longer stops unrelated tasks, while the task and conversation that are really at risk stay protected ## Linked Issues or Issue Description What happened? After two hard control-plane restarts about 20 s apart, one Codex session close became unrecoverable. From then on, every Codex execution in the company failed at admission with `native_session_cleanup_quarantined: prior session cleanup requires operator recovery…` (`retryQuarantinedSessionCleanups`). Board Retries for four unrelated tasks were accepted and then failed at once. Steps to reproduce 1. Run two native Codex tasks in one company (local runner). 2. Make one session's close throw `NativeSessionCloseUnrecoverableError` (in the wave: a protocol failure after a second hard kill, then an unsettled runner suspension). 3. Start or retry a different Codex task in the same company. Expected behavior Only work that shares a resource with the retained owner waits. Other tasks run normally. Actual behavior Every later Codex run in the company fails within about 1 s, until the server process restarts. ## What Changed - `packages/paperclip-runner/src/native-session-runtime.ts`: the cleanup domain is now `{ key, resources }`. `key` is the old backend key (company, backend kind and name, optional remote cleanup scope). `resources` are the session's issue and its normalized session (or its run when it has no normalized session). - The admission gate (`retryQuarantinedSessionCleanups`) and its retained close owners block a new session only when the backend key matches and at least one resource matches. - The exact retirement paths (`completeRetainedNativeSessionCleanup`, `completeTerminatedRemoteNativeSessionCleanup`, `completeTerminatedLocalNativeSessionCleanup`) read the unchanged backend key, so their behavior does not change. - New regression test: an unrecoverable close on session A does not block session B of another task. A session with the same issue, or with the same normalized session, stays blocked. Why this scope is safe: the resources that a retained owner can still hold are its runner state root and its workspace. The server derives the runner state root from the session scope (company, agent, workspace, provider, normalized session). The workspace belongs to the task. Different tasks already run concurrently on the same backend, so a different task shares no resource with the retained owner that normal concurrency does not already share. ## Verification - Red before the fix: `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/native-session-runtime.test.ts -t "unrelated tasks of the same backend"` failed with `native_session_cleanup_quarantined` for the unrelated task (the RC-20 error). - Green after the fix: the same test passes. The full file passes (211/211). - `pnpm --filter @paperclipai/paperclip-runner typecheck` passes. ## Risks - A retained owner can no longer block a different task that shares a project workspace. Concurrent tasks in a shared workspace are already allowed, so this adds no new kind of overlap. - The same task and the same conversation stay blocked. A Retry of the task whose close failed can still be accepted and then fail at admission. The trigger of that close is handled separately. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:39:31 -06:00
DottaandClaude Opus 5.5	07ddf9a5c2	fix(native): keep accepted results when runner close cannot prove suspension (#19525 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs persist an accepted result first. The Runner then closes and suspends its session. > - Under heavy host load, the Runner close can fail to prove a durable suspension after the result is accepted. > - The executor then kept its finalization lease. The heartbeat finalizer got `native_finalization_lease_busy`, and the run ended `failed/adapter_failed`. > - The provider work was complete. Only Runner bookkeeping failed, so the run must not fail. > - This pull request hands the exact lease to finalization and completes the run from its accepted result. > - The benefit is that an accepted result stays accepted, independent of host load. ## Linked Issues or Issue Description What happened? A native ACPX Claude run asked a structured question. The control plane accepted its result (`Waiting for…`) and recorded `run.terminal` with `runTerminalState=succeeded`. The Runner close then took about 8.6 s. Its `runner.suspend` command ended `failed`, but the durable Runner state was `suspended` with a matching identity. The close threw `NativeSessionCloseUnrecoverableError`. The executor emitted `native.post_result_failure` (`category: runner_settlement_incomplete`, `resultAlreadyAccepted: true`) and rethrew `native_session_lease_lost`. Its finalization lease stayed held (its TTL is 20 minutes). `finalizeNativeRun` got `native_finalization_lease_busy`, and the run ended `failed/adapter_failed`. No restart was in progress. Host load average was about 40. Expected behavior After the control plane accepts a result, finalization bookkeeping must not fail the run. The run settles from its accepted result. Repair of the Runner checkpoint stays with the existing next-run admission rules. Steps to reproduce Run `pnpm exec vitest run src/services/heartbeat/runtime-execution.test.ts -t "accepted local result"` in `server/`. The test uses the real executor, the real control-plane result commit, and embedded PostgreSQL. The provider boundary accepts a result and then throws `NativeSessionCloseUnrecoverableError`. On the base revision, the finalization row keeps the executor lease (`phase: workspace_finalizing`) and the finalizer reports `lease_busy`. Paperclip version or commit Base `ff1d1dea5d`. The incident was on `819420a60a`. ## What Changed - `native-session-executor.ts`: after the rejected failure write proves an accepted result under the exact, unexpired lease, the executor releases that lease to finalization. It uses the same exact-authority predicates as the normal release. Before, the lease stayed held until its TTL expired. - `runtime-execution.ts`: for `runner_settlement_incomplete` on a local workspace (no workspace sync and no remote workspace authority), Paperclip records `workspace_finalize` as succeeded, finalizes the accepted result, and returns it to the ordinary completion path. - New `native-accepted-result-settlement.ts`: rebuilds the adapter result from the durable accepted result row, its terminal, the run's native session, and the accepted usage checkpoint. The result carries the closed `nativePostResultFailure` diagnostic. It contains no provider text. - Other post-result categories (for example `io_not_found`) keep their workspace-failure handling. They now also reach finalization at once, instead of after the lease TTL. - `doc/execution-semantics.md`: describes the settlement. ## Verification - Red, then green: `settles an accepted local result when runner close cannot prove suspension` and `keeps workspace failure handling for other accepted-result failures on a local workspace` in `server/src/services/heartbeat/runtime-execution.test.ts`. Both failed on the base revision with the executor lease still held. Both pass with this change. - Safety case `settles an accepted local result ... (workspace-check-failed)`: when the `workspace_finalize` check fails, the run keeps the existing failure path. Case `terminal-unreadable`: when the accepted result cannot be rebuilt, only a failed `workspace_finalize` record is written. - `pnpm exec vitest run src/services/heartbeat/runtime-execution.test.ts`: all pass (some cases time out at 30 s only under host load above 40; they pass on a narrow rerun). This includes the 14 existing post-result diagnostic and authority cases (expired, replaced owner, generation, attempt, boot, pid, process start, stopped, lock busy, and others). - `pnpm --filter @paperclipai/server typecheck`: pass. - Live cell `local-session-integrity.runner-acpx-claude.local.structured-question-restart-resume` (real ACPX Claude, local runner): 3 runs at host load average about 40. Runs 1 and 2 passed. Run 3 hit the fault naturally: the first run emitted `native.post_result_failure` (`runner_settlement_incomplete`) and now ended `run succeeded` (before this change it ended `failed/adapter_failed`). The cell then failed later, in the resume run after the answer: `runner_state_identity_mismatch: prior_owner_terminal_state_indeterminate; retirement_refused=target_not_sandbox_runner`. That is a separate follow-up (see Risks). ## Risks - A changed lease handoff. The executor releases the lease only after the rejected failure write proves an accepted result under the exact owner, attempt, controller generation, controller identity, and an unexpired lease. A stale or replaced owner does not release anything. A failed release is bounded and best effort. The old behavior then applies. - The run now completes as `succeeded` while the local Runner checkpoint is not proven suspended. The next run on the same session then meets the existing fence: the continuity-break retirement (`retireFinishedPerTurnRunnerdOwner`) accepts only sandbox Runners, so a local Runner is refused with `target_not_sandbox_runner` and its state root is quarantined. This PR does not change that fence. Extending the retirement to local Runners needs its own proof that the local Runner process stopped, and is a follow-up. - Settlement happens only when the accepted result and its terminal are rebuilt first, and the `workspace_finalize` record then succeeds. A failed branch or plugin workspace check keeps the failure path. - Synced and remote workspaces are not changed. They keep their copy-back retry path. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test runs). Extended reasoning was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #19126 added the closed post-result diagnostic that this change uses. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:36:34 -06:00
Nicky LeachandClaude Opus 5.5	1206373a95	fix(environments): close Apex findings on managed plugin environments (#19526 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document, and #19508 added plugin environment entries to it > - Greptile's Apex review of #19508 arrived after it merged, with three findings > - A credential inside an array nested in an array passed the secret check and the response redaction > - Recovery of an archived environment listened on one worker handle, which a restart replaces, and its attempts were not tracked through idle checks and shutdown > - This pull request closes all three findings > - The benefit is that credentials never pass the check, an environment comes back after any worker restart, and shutdown never cuts off a recovery write ## Linked Issues or Issue Description Refs #19508 (the Apex findings this pull request fixes) Subsystem affected Server environments: the managed-config parser, the environment routes' response floor, the managed environment bootstrap, and the plugin worker manager. Problem or motivation Greptile's Apex review of #19508 found three problems. (1) `findSecretLikeConfigKey` and `redactSecretLikeConfigKeys` skip array elements that are arrays, so `{"groups":[[{"apiToken":"..."}]]}` passes the boot check and is echoed to restricted readers. (2) Recovery subscribes to `ready` on one worker handle. A restart such as a settings save replaces that handle, so the new worker's `ready` never reaches the listener, and the environment stays archived. (3) A recovery attempt is fire-and-forget. The worker manager's work token ends when the worker replies, but the database write continues, so an idle check or the database shutdown can overlap it. Proposed solution Walk arrays at any depth in both functions. Add a manager-wide `onWorkerReady` subscription and use it for recovery. Run every recovery attempt through a controller that tracks it as idle work and that shutdown stops and awaits before the database closes. Alternatives considered Re-subscribing on each new handle after a restart. That needs a hook in every restart path. The manager-wide subscription covers all of them in one place. Roadmap alignment This only fixes code that #19508 added or touched. It adds no product surface. ## What Changed - `server/src/services/managed-config.ts`: `findSecretLikeConfigKey` walks arrays at any depth (new `findSecretLikeConfigKeyInValue`). This covers sandbox `config` and plugin `driverConfig`. - `server/src/routes/environments.ts`: `redactSecretLikeConfigKeys` walks arrays at any depth (new `redactSecretLikeConfigValue`). - `server/src/services/plugin-worker-manager.ts`: new optional `onWorkerReady(listener)` on `PluginWorkerManager`. It fires on every worker `ready`, including a replacement handle from a restart. A listener error is logged and does not affect the worker. - `server/src/services/managed-environments.ts`: - Sandbox and plugin recovery subscribe through `subscribeWorkerReady`. It prefers `onWorkerReady` and falls back to the handle when the manager has no such method. - New `createManagedEnvironmentRecovery()` controller. Each attempt runs inside `trackIdleWork` for its whole duration. `stop()` removes every `ready` subscription, refuses new attempts, and awaits unfinished ones. - `server/src/index.ts`: creates the controller, passes it to `applyManagedEnvironments`, and awaits `stop()` during shutdown, after the delivery workers stop and before the application teardown and database close. ## Verification - `cd server && npx vitest run src/__tests__/managed-config.test.ts`: a secret-like key in an array nested in an array refuses boot, for plugin `driverConfig` and sandbox `config`. - `npx vitest run src/__tests__/environment-routes.test.ts`: a nested-array credential is removed from the response, and other nested values stay. - `npx vitest run src/services/managed-environments-plugin-entries.test.ts`: - With `onWorkerReady`, recovery never consults the handle. It ignores another plugin's `ready`, re-ensures on this plugin's `ready`, and then unsubscribes. - An attempt counts in `idleWorkSnapshot()` until its ensure settles. `stop()` unsubscribes, waits for the unfinished attempt, and later `ready` events start nothing. - `npx vitest run src/services/managed-environments.test.ts src/__tests__/managed-plugin-environments.test.ts`: pass. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - The redaction now also walks nested arrays in sandbox config responses. Only credential-shaped keys are removed; other values are unchanged. - The boot check now refuses a document that hides a secret-like key in a nested array. A valid document has no such key. - `onWorkerReady` is optional, so other `PluginWorkerManager` implementations and test fakes keep working. - Shutdown now waits for an unfinished recovery attempt. An attempt is one worker call plus one short transaction. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:35:57 -08:00
Nicky LeachandClaude Opus 5.5	3b3037aab5	fix(ci): run the sandbox provider tests in required CI (#19533 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox provider plugins (Daytona, E2B, Cloudflare, Kubernetes, Modal, and others) run agents in remote sandboxes > - The providers are outside the root pnpm workspace, so `pnpm test:run` and the CI test lanes never run their Vitest suites > - paperclipai/paperclip#18839 typechecks those tests, but a test that fails at runtime still fails nothing > - The providers have about 1,040 tests, and they finish in about 15 seconds on Linux > - This pull request adds a required `Sandbox provider tests` job that runs them when a change can affect them > - The benefit is that a broken provider, plugin SDK, or shared contract fails CI on the pull request instead of in a release or a paid end-to-end run ## Linked Issues or Issue Description Refs #18839 What happened? No CI job runs the sandbox provider tests. `runner-full-stack-e2e.yml` builds the Daytona plugin for a paid nightly run, but it does not run any provider's `pnpm test`. Expected behavior A pull request that can break a provider runs that provider's tests, and a failure blocks `ci / verify`. Steps to reproduce Add `expect(1).toBe(2)` to a test in `packages/plugins/sandbox-providers/e2b/src`. Open a pull request. CI stays green. ## What Changed - Added `scripts/test-sandbox-providers.mjs` and its tests. The script installs the providers through `typecheckTestPrograms` with no programs, because that function already owns the provider install and the plugin SDK link. Then it runs each provider's `pnpm test`, four at a time, and prints each provider's output as one block. It fails if a suite fails or if no provider has a test script. - The script skips `cloudflare/bridge-template`, because Cloudflare's Vitest config already runs those tests. - Added the `sandbox_provider_tests` job to `.github/workflows/pr-trusted.yml`, and made `ci / verify` require its result. - Added the `//#ci:sandbox-providers` task to `turbo.json` and its lane to `.github/ci-task-matrix.json`. The CI planner runs the job only when a change touches `packages/plugins/sandbox-providers/`, `packages/plugins/sdk/`, or `packages/shared/`. `packages/shared` is the plugin SDK's only workspace dependency. Global CI inputs, such as `scripts/`, `.github/*`, `pnpm-lock.yaml`, or any `package.json`, still select every job. - The job turns off core dumps. A Daytona capacity test kills a process at a file-size limit inside the exported directory. If core dumps are on, the core file joins the export, and the test sees 12 files instead of 11. - Updated the job lists in `scripts/__tests__/e2e-shard.test.mjs` and the job count in `.github/scripts/tests/ci-profile.test.mjs`. Added planner tests to `.github/scripts/tests/ci-plan.test.mjs`. One of these tests fails if the plugin SDK gets a new workspace dependency that is not in the job's inputs. - Added the new test file to `test:release-registry`, and documented the script in `doc/DEVELOPING.md`. ## Verification - `node scripts/test-sandbox-providers.mjs` on macOS: 8 of 9 suites pass in 31.6 s, including the install. Kubernetes fails 12 `file-sync` tests, because macOS has no `/proc/self/fd`. Some Daytona tests skip, because they need GNU `tar` and coreutils. - The same script in a Linux container (`node:24`, a non-root user, core dumps off): all 9 suites pass, with 1,012 tests passed and 26 skipped. The script took 15 s after the install. - Kubernetes: 205 passed. - Daytona: 590 passed and 16 skipped. - With core dumps on, Daytona's `preserve literal capacity repair` test fails. The export includes a `core` file. With `--ulimit core=0`, it passes. - To look for timing failures under load, I ran the whole script 12 more times in the container, limited to 2 CPUs (8 runs) and 1 CPU (4 runs). All 12 runs passed. They took 45 to 76 s on 2 CPUs and 73 to 102 s on 1 CPU. - `ubuntu-latest` has `zstd`, and my container did not, so some Daytona tests skipped in those runs. With `zstd` installed, Daytona passed with 598 tests and 8 skipped, and only the live tests skipped. One earlier `zstd` run failed while the stress runs used the same CPUs, and I did not capture its output. The next run passed. - `node --test` on every test that reads `pr-trusted.yml`, the CI planner, or the task matrix: 816 passed. These include `.github/scripts/tests/.test.mjs`, `scripts/__tests__/e2e-shard.test.mjs`, `scripts/ci/test-selection-workflow.test.mjs`, and the new tests. - `pnpm run test:release-registry` on macOS: 251 of 255 pass, including the new tests. The 4 failures are `wait_for_npm_package_versions` tests in `scripts/release-lib.test.mjs`. They fail the same way on master on the same machine, and CI runs this suite on Linux. - `actionlint` 1.7.7 reports no errors on the changed workflow. - I called `planTasks` from `.github/scripts/ci-plan.mjs` with one changed file at a time. `server/src/index.ts` and `ui/src/App.tsx` leave `sandbox_provider_tests` off. `packages/plugins/sandbox-providers/e2b/src/plugin.ts` and `packages/shared/src/index.ts` turn it on. This pull request cannot run its own new job. `pr.yml` calls `pr-trusted.yml@master`, so the job first runs on pull requests after this one merges. After merge, check the first pull request that changes a provider or `packages/shared`. ## Risks - First real run is after merge. I measured the time on Linux in Docker, not on a CI runner. CI runners are about 2 times slower than my machine, so I expect about 2 to 3 minutes for the job, including the dependency restore. The job runs in parallel with the other jobs, and the slowest required job takes about 13 minutes. So the job should not make CI slower. - Open provider pull requests. After this merges, open pull requests that change a provider run its tests. If those tests fail, `ci / verify` blocks them until they are fixed. - `packages/shared` changes often. Any change there runs this job. A Linux-only provider failure would block those pull requests. All suites pass on Linux today. - One unexplained Daytona failure. One Daytona run with `zstd` failed under CPU contention, and I did not capture its output. Some Daytona tests have short deadlines. If the job is flaky after merge, quarantine the failing test or revert the job, and do not leave `ci / verify` red. - Path rules can be too narrow. If a provider starts to import another workspace package, a change to that package does not run this job. The plugin SDK dependency test covers the SDK. It does not cover a provider that imports a workspace package directly. Today no provider does. - Overlap with paperclipai/paperclip#19489 and paperclipai/paperclip#19522. Both pull requests change the provider install in `scripts/typecheck-test-programs.mjs`. This pull request does not edit that file. It calls the same install function, so it uses their install changes when they merge. All three pull requests edit the same section of `doc/DEVELOPING.md`, so a small text conflict is possible. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with tool use (shell, file edits, GitHub CLI, Docker). Extended reasoning was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:34:03 -08:00
DottaandClaude Opus 5.5	874fed09a4	test(restart-chaos): enable isolated workspaces for repo tasks (#19536 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart chaos harness (`tests/restart-chaos`) restarts the control plane while agents work, then grades the result > - Rule R2 says files an agent wrote before a restart must still be present afterwards > - Repo workloads failed R2 with `pre_fault_files_lost`. The first run worked in the shared project checkout, not in its own worktree > - The seed sets a `git_worktree` project policy, but the server applies that policy only when the instance `enableIsolatedWorkspaces` setting is on. The seed did not turn it on > - This pull request makes the seed turn on the setting (as users do on the Experimental page, and as `tests/runner-e2e` does) and fail if the server does not confirm it > - The benefit is that repo tasks get their own worktree, so one task's files cannot appear in another task's R2 snapshot ## Linked Issues or Issue Description Refs #19477 What happened? In a restart chaos wave, two repo tasks (B1 and B5) ran at the same time. Both ran in the project primary checkout (`strategyType: project_primary`). Task B1's `NOTES-.md` file and its `cli.py` edit appeared in the B5 task's workspace before any fault. After a Retry, the B5 task moved to a fresh attempt directory, and R2 reported the B1 file as lost from B5. Expected behavior* Each repo task gets its own git worktree, as the harness project policy asks. The R2 snapshot of a task contains only that task's files. Steps to reproduce 1. Run `tests/restart-chaos/main.ts --env local --adapters codex --workloads B1,B5 --concurrency 2`. 2. Read `currentExecutionWorkspace` for each task. It shows `project_primary` and the fixture checkout as `cwd`. 3. Read `GET /api/instance/settings/experimental`. `enableIsolatedWorkspaces` is `false`. Root cause `server/src/services/heartbeat.ts` reads `enableIsolatedWorkspaces` and `gateProjectExecutionWorkspacePolicy` (`server/src/services/execution-workspace-policy.ts`) returns `null` when the setting is off. The project policy is then ignored, and the task resolves to the shared primary checkout. `tests/restart-chaos/seed.ts` created the project policy but did not turn on the setting. ## What Changed - `tests/restart-chaos/seed.ts`: before it creates the project, the seed sends `PATCH /api/instance/settings/experimental` with `enableIsolatedWorkspaces: true`. If the response does not confirm the setting, the seed stops with an error. It records a `seed_isolated_workspaces` timeline entry. - `tests/restart-chaos/seed.test.ts`: new unit tests. One test checks that the seed turns on the setting before it creates the `git_worktree` project. One test checks that the seed refuses to continue when the server does not confirm the setting. - `tests/restart-chaos/README.md`: the `seed.ts` row says that the seed turns on the setting, and why. ## Verification - Red before the fix: `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts seed.test.ts` failed 2/2 (`expected -1 to be greater than or equal to 0`). - Green after the fix: the full restart-chaos unit suite passes, 5 files and 42 tests. - Live wave on this branch (`16f5c8c5d7`, local runner, native Codex, macOS, load average about 40): `tests/restart-chaos/main.ts --env local --adapters codex --workloads B1 --tasks 3 --concurrency 1 --faults K1 --seed 21` (wave `w-20261106T181436-s21`). The timeline shows `seed_isolated_workspaces`. Result: 3/3 pass. Each task crossed a SIGKILL restart. R1, R2, R4, R5, R6 and the workload oracle passed for each task, with no lost files. - Each task's pre-fault file list now has a `.git` file (a git worktree, not the primary checkout) and only its own `NOTES-.md` file. In the failing wave, the B5 task's list had the B1 task's `NOTES-.md` file. - Not exercised: no task needed a Retry in this wave, so the Retry-to-fresh-attempt path did not run. - Ledger preflight: install and runner builds passed. The provider-free restart suite had 2 failures in `runnerd-codex-transport-warm-attachment.test.ts` at load average 44; a rerun of that file passed 18/18. ## Risks - Low risk. The change is in the test harness only. It changes no product code. - The harness instance is temporary, so the experimental setting does not reach a real instance. - Repo tasks now run in git worktrees. Earlier waves ran in the shared checkout, so R2 results for repo workloads from earlier waves are not comparable with new waves. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). Standard context window; reasoning enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:33:13 -06:00
Devin FoleyandPaperclip	e3d8f42e19	fix(ui): recover stalled onboarding connection checks (#19505 ) Bound the complete onboarding connection check to two minutes, abort its requests on timeout, and ignore late results. Keep the saved account available for retry and resume its matching unfinished setup after refresh. Verified 242 focused tests, workspace typecheck and build, and the complete hosted CI check set. Greptile is 5/5 with no unresolved review comments. Local full-UI timing failure and interrupted CI runner attempts remain documented in the PR evidence. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-06 10:31:18 -08:00
DottaandClaude Opus 5.5	beaa9a622d	fix(native): re-bootstrap a runner killed before session.open (#19524 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs start a local `paperclip-runnerd` process. The control plane keeps a durable state root for each session. > - When the control plane restarts, restart recovery classifies each live native run. A run whose runner died before any provider identity existed gets the `bootstrap_incomplete` disposition. Recovery then quarantines the old state root and starts the bootstrap again. > - The executor accepts that disposition only when the durable state root proves that no provider work started. The old proof required `connectionCount === 0` and every command `pending`. > - A restart chaos wave killed the server about 2 s after runnerd started. The runner had connected and completed `run.prepare`, but `session.open` was still pending. No event was committed, so no provider session or turn existed. > - The proof rejected that root. The run failed with `runner_state_identity_mismatch: bootstrap_not_proven_incomplete`, and the task stopped with `native_continuation_requires_reconciliation`. Nothing continued automatically. > - This pull request widens the proof to the exact window where provider work cannot have started, and keeps every other shape fail-closed. > - The benefit is that a restart during native runner bootstrap continues automatically for every native provider. ## Linked Issues or Issue Description What happened? A control plane restart that lands after the native runner connected and acknowledged `run.prepare`, but before `session.open` completed, fails the run with `adapter_failed` and `runner_state_identity_mismatch: bootstrap_not_proven_incomplete`. Startup recovery chose `bootstrap_incomplete` (controller generations 2 and 3), but the executor proof in `migrateRunnerdStateRootForExecution` refused it. The durable control-plane state had `connectionCount: 1`, `run.prepare` completed, `session.open` pending, no committed events, and `ackedSourceSeq: 0`. The runner journal had `nextSourceSeq: 1`, an empty outbox, and the provider state was `prepared` with no provider session ID. Steps to reproduce 1. Start a native task (seen with OpenCode on a local runner; the window applies to every native provider). 2. Kill the control plane about 2 s after runnerd starts, after `run.prepare` completes and before `session.open` completes. 3. Let the control plane restart. Expected behavior Recovery quarantines the state root and starts the bootstrap again. The task continues without a user action. Actual behavior The run fails and the task is blocked with `native_continuation_requires_reconciliation`. ## What Changed - `runnerdStateIsIncompleteBootstrap` (native-session-executor.ts) now also accepts a connected root when all of these are true: no committed event, `ackedSourceSeq === 0`, only `run.prepare` and `session.open` commands, `session.open` not completed, `run.prepare` pending or completed, no warm transition or staged attachment, and the bootstrap was not adopted. - When `runner/runner-state.json` exists, the runner journal must agree: empty outbox, `nextSourceSeq === 1`, `ackedSourceSeq === 0`, and no completed `session.open` or other processed command. An unreadable runner journal stays fail-closed. This covers events that the runner emitted but the controller never committed. - The remote "claim an untouched session" path keeps the old never-connected proof (`runnerdStateIsUnconnectedBootstrap`), so its behavior does not change. - Tests: a red test with the production state shape (connected, `run.prepare` completed, `session.open` pending), plus fail-closed counter-tests for a completed `session.open`, committed or acknowledged events, a turn command, a failed `run.prepare`, an adopted bootstrap, a warm transition, an uncommitted runner event, a runner-completed `session.open`, a runner-processed turn, and a torn runner journal. ## Verification - Red before the fix: `pnpm exec vitest run src/services/native-runtime/native-session-executor.test.ts -t "native incomplete-bootstrap evidence"` failed on "allows a fresh bootstrap after connect and run.prepare when session.open never completed" (`expected false to be true`). Green after the fix: 13/13. - The proof run against the kept durable state root from the failing wave (read only) returns `true` after the fix. - Related suites: `native-session-executor.test.ts` and `native-restart-recovery.test.ts` filtered to `bootstrap\|restart\|remote\|unstarted\|untouched`: 301 passed. - `tsc --noEmit -p server/tsconfig.json` passes. - Live restart chaos proof: see the comment on this PR. ## Risks - The proof now accepts a root where the runner received `session.open` but did not complete it. The runner can have started a provider process or created an empty provider session in that window. Recovery only proceeds when restart classification already proved that the runner process and its group are dead and no known provider process is alive. No provider turn can exist, because the controller never issued a turn command. The worst case is an orphaned empty provider session, not repeated work. - Every state with committed or acknowledged events, a completed `session.open`, a turn, a warm transition, an adopted bootstrap, or an unreadable runner journal still fails closed. - Out of scope: when the proof fails legitimately, the task offers the existing execution-blocker Retry. In the failing wave the issue interruption also showed a budget wait (`userAction: budget`). That is the RC-06 accounting issue and belongs to a separate fix. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use (shell, file edits, test execution). Default context window. Extended reasoning was available. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:30:50 -06:00
DottaandClaude Opus 5.5	a6c2280630	fix(ui): dock the announcement card in the sidebar so it never covers the task thread (#19511 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A new user lands on their first task after onboarding. The task thread and the composer are the core of that screen. > - The app shows an optional in-app announcement (now "Introducing Connectors"). On desktop the card floats at the bottom-left, 22rem wide, but the sidebar is only 240px wide. > - In the Nov 5 cloud onboarding test, the tester said: "onboarding popup blocks core real estate". > - At 1280x800 the card covers the left part of the composer. At 1440x900 and 1280x800 it covers the task thread area and the sidebar account menu. > - This pull request docks the card inside the desktop sidebar column, in normal layout flow, above the account menu. > - The benefit is that the card stays visible and dismissible, but it can never cover the thread or the composer. ## Linked Issues or Issue Description What happened? On a new company's first task, the "Introducing Connectors" announcement card floats over the task thread. At 1280x800 it covers the left part of the composer. Steps to reproduce 1. Create a new company and open its first task at 1280x800. 2. Wait 3 seconds for the announcement card to show. 3. See the card on top of the composer and the account menu. Expected behavior Optional announcements do not cover the task thread or the composer. Evidence The staging test company (Claude subscription path) has a local dismissal for `2026-09-14-connectors`. The live feed at `pages.paperclip.ing/announcements/v1/current.json` serves that announcement. No other first-use overlay showed on the first task or the dashboard. Toasts and the task-detail error notice are transient and are not first-use popups. ## What Changed - `AnnouncementWell` has a `placement` prop: `sidebar` (in-flow, no `fixed`) or `floating` (the earlier fixed placement). - `Layout` renders the `sidebar` placement inside the desktop `SidebarShell`, above the account menu. It renders the `floating` placement only on mobile, where it stays above the bottom nav. - While the desktop sidebar is collapsed to a rail or closed, the card waits. It shows again when the sidebar is expanded. - In the sidebar, the card is capped at half the window height (`--announcement-sidebar-max-height: 50dvh`) and scrolls. It uses the existing compact image height token, so the sidebar navigation keeps most of its height. - New e2e spec `tests/e2e/announcement-placement.spec.ts`. It makes a new company with a team lead and a first task. It serves the live connectors announcement through the app's announcement API. At 1440x900 and 1280x800 it checks that the card does not overlap the composer or `#main-content`, that the card is fully in the viewport and dismissible, and that it does not overlap the account menu trigger and the Account dialog opens. - Unit test for the two placements in `AnnouncementWell.test.tsx`. Product decision for Dotta: this PR does not hide announcements on first use. Another option is to hold all announcements until the user has used the product for some time (for example, after the first task is done or after the first day). That is a one-line gate in `AnnouncementWell`'s `enabled` condition, if we want it. ## Verification Red on `master` (`819420a60a`), same spec, before the fix: ``` [1/1] [chromium] › tests/e2e/announcement-placement.spec.ts › Announcement placement on a new company's first task › the announcement does not cover the task thread or composer Error: 1440x900: announcement overlaps task thread Expected: 0 Received: 42904 Error: 1280x800: announcement overlaps composer Expected: 0 Received: 8568 Error: 1280x800: announcement overlaps task thread Expected: 0 Received: 42904 1 failed ``` The spec also checks the account menu (requested by the N-13 connectors-upsell fix). Red on `master` with the same spec: ``` Error: 1440x900: announcement overlaps task thread Received: 42904 Error: 1440x900: announcement overlaps account menu Received: 5312 Error: locator.click: Test ended. (the card intercepts the "Open account menu" click) ``` Green after the fix: ``` [1/1] [chromium] › tests/e2e/announcement-placement.spec.ts › Announcement placement on a new company's first task › the announcement does not cover the task thread or composer 1 passed (45.0s) ``` Review follow-up (Greptile/Ponytail: a long announcement in a 208px sidebar had no height cap). A second spec uses a 400-character description, a 208px sidebar, and a 1280x720 window. Red without the cap: ``` Error: docked card keeps at most half the window height Received: 732 Error: expect(locator).toBeInViewport() failed (sidebar "Tasks" link) Received: viewport ratio 0 Error: expect(locator).toBeInViewport() failed (account menu) Received: viewport ratio 0 ``` The spec sets the sidebar width key for both the classic and the Coordinator workspace, and asserts the rendered width is 208px. Green with the cap (both specs): ``` 2 passed (57.8s) ``` Neighbouring suites: ``` cd ui && npx vitest run src/components/AnnouncementWell.test.tsx src/components/AnnouncementCard src/hooks/useAnnouncement.test.tsx src/components/Layout.test.tsx Test Files 4 passed (4) Tests 57 passed (57) ``` - `pnpm check:token-gates`: All gates clean. - `pnpm --filter @paperclipai/ui typecheck`: pass. - Screenshots: the spec writes `announcement-1440x900.png` and `announcement-1280x800.png` to its test output. Before: the card (352px wide) sits over the composer at x 296–364 and over the account menu. After: the card is 216px wide inside the sidebar, above the account menu. The thread and composer are clear. Run it: ``` npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/announcement-placement.spec.ts ``` ## Risks - Low risk. The card now takes vertical space in the sidebar while it shows. The sidebar navigation scrolls as before. The card goes away when the user dismisses it. - Users who keep the desktop sidebar collapsed do not see the announcement until they expand the sidebar. - Mobile behavior does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use (shell, browser, Playwright). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> canary/v2026.1106.0-canary.18	2026-11-06 12:19:46 -06:00
Nicky LeachandClaude Opus 5.5	35280516c0	fix(ci): set up pnpm in the Vitest selection job like other trusted jobs (#19527 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Trusted CI workflows set up pnpm in a fixed way. Node comes first, and npm audit, fund and update-notifier calls are off. > - A policy test in the Runner support suite checks every pnpm setup in `pr-trusted.yml` for this shape. > - PR #18348 added a pnpm setup to the new Vitest selection job. That setup did not have the npm overrides, and setup-node was not directly before it. > - PR CI does not run the Runner support suite. So the policy test failed on master, and nobody saw it. > - This pull request gives the selection job the same pnpm setup as the other eight jobs. > - The benefit is that every trusted pnpm setup follows one rule again, and the policy test passes. ## Linked Issues or Issue Description Refs #18348 (added the job). Refs #18625 (its description noted the "nine bootstraps where eight are expected" failure). Related: #19229 also edits the pnpm steps in `pr-trusted.yml`. What happened? `pnpm test:e2e:runner:unit` fails on master in two tests: "keeps pnpm bootstrap registry telemetry out of trusted workflow setup" and "installs a modern Node runtime before every trusted pnpm bootstrap". `pr-trusted.yml` has nine pnpm setups. The ninth, in `test_selection`, has no `NPM_CONFIG_AUDIT`, `NPM_CONFIG_FUND` or `NPM_CONFIG_UPDATE_NOTIFIER` override. The step directly before it is the eligibility check, not setup-node. Expected behavior Every trusted pnpm setup has the three overrides, and setup-node is the step directly before it. Steps to reproduce Run `pnpm test:e2e:runner:unit` on master. Read the two failures above. ## What Changed - `pr-trusted.yml`, job `test_selection`: add "Setup Node.js for pnpm bootstrap" directly before "Setup pnpm". Both steps use the existing eligibility condition. The pnpm step now sets the three npm overrides. Both actions keep their pinned SHAs, with version comments added. - `tests/runner-e2e/workflow-security.test.ts`: the expected count of pnpm setups is now set per workflow: 8 for `runner-full-stack-e2e.yml` and 9 for `pr-trusted.yml`. The checks on each setup do not change. ## Verification - On this branch, the two pnpm tests in `workflow-security.test.ts` pass. Before this change, they failed with "expected 8, got 9". - The other three failures in that file are fixed separately in the companion PR that repairs the rest of the suite. - `node --test scripts/ci/test-selection-workflow.test.mjs`: 4 passed. - The workflow parses as YAML. The `test_selection` steps are now: checkout, setup-node, eligibility check, setup-node for pnpm, setup pnpm, restore dependencies, select, upload, publish. - `pr.yml` calls `pr-trusted.yml@master`. So this PR's own CI runs the master copy of this workflow. The change runs in CI only after merge. ## Risks - Low risk. The new setup-node step runs again with the same Node 24 version. It adds about one second when selection is eligible. When selection is not eligible, both steps are skipped, as before. - `test_selection` has `continue-on-error: true`. If the new steps fail, the original test matrix still runs. - #19229 removes `version: 9.15.4` from every pnpm step in this file. If both PRs merge, the second one needs a small rebase on this step. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), by Anthropic, in Claude Code with tool use (shell, file edits, git, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:14:44 -08:00
Devin FoleyandPaperclip	b85557a122	Fix OpenAI subscription waiting labels (#19521 ) Show “Waiting for auth…” while OpenAI subscription sign-in completes in onboarding and the shared provider connection screen. Preserve Claude’s code-entry label and align connection previews and existing regression expectations. Verified 246 component tests, workspace typechecks, the full production build, token gates, and all 54 applicable CI checks. Greptile scored the change 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-06 10:12:12 -08:00
github-actions[bot]andlockfile-bot	17b4f972bf	chore(lockfile): refresh pnpm lockfiles (#19534 ) ## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description What happened? Dependency manifests on master changed after the last committed lock refresh. Expected behavior Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. Steps to reproduce Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Refresh the sandbox provider lockfiles from their manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>	2026-11-06 18:09:43 +00:00
Nicky LeachandClaude Opus 5.5	e3041f5481	fix(ci): let the lockfile bot own the sandbox provider lockfiles (#19522 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The repository has a supply-chain rule: contributors do not commit lockfiles. A bot and verified Dependabot commits write them. > - The rule checks only the root `pnpm-lock.yaml`. > - The sandbox providers are outside the root workspace, so they have two more pnpm lockfiles. Pull requests edited them by hand, for example #12791, #18839 and #19489. > - This pull request makes the same rule and the same bot cover those two lockfiles. > - The benefit is one lockfile rule for every managed pnpm lockfile, and no lockfile edits in contributor pull requests. ## Linked Issues or Issue Description Refs #19489 What existing behavior does this improve? The lockfile policy and the lockfile refresh bot. Subsystem affected CI: `.github/scripts/lockfile-trust.mjs`, `.github/workflows/refresh-lockfile.yml`, and the sandbox provider installs. Current behavior `lockfile-trust.mjs` blocks contributor edits to `pnpm-lock.yaml` only. A contributor can edit `scripts/typecheck-sandbox-providers/pnpm-lock.yaml` or `packages/plugins/sandbox-providers/daytona/pnpm-lock.yaml`, and CI passes. The refresh bot does not update these files. `typecheck:test-programs` fails on a stale provider lockfile and tells the contributor to commit a new one. Proposed behavior The policy blocks contributor edits to all three lockfiles. The refresh bot regenerates all three after merge. CI resolves a stale provider lockfile in place for the run, as it does for the root lockfile. Reason and benefit Lockfile content comes only from the bot or from verified Dependabot commits. Contributors do not need to run pnpm in special folders. Breaking changes None. A pull request that edits a provider lockfile now fails the policy check. Remove the lockfile change from the pull request. The bot commits it after merge. ## What Changed - `lockfile-trust.mjs` now exports `LOCKFILES`: the root lockfile, the shared sandbox provider lockfile, and the Daytona lockfile. The policy blocks contributor edits to each of them. A Dependabot pull request must match a verified Dependabot commit for each lockfile that it changes. The `merged` command and the reporting-lock proof still use only the root lockfile. - `check-pr-lockfile.mjs` (advisory check) uses the same list. - `refresh-lockfile.yml`: - A job-level `LOCKFILES` env lists the same three files. A test checks that it equals the policy list. - A new step regenerates the two provider lockfiles with `--resolution-only --ignore-scripts --no-frozen-lockfile`. - The unexpected-changes check, the change check, and `git add` cover all three lockfiles. - The release-age gate checks the added versions in every lockfile. - A held refresh saves all three lockfiles at their repository paths, with a `source-locks.sha256` file. The next run reuses them only if every master lockfile is unchanged. An artifact in the old format is not reused. - The PR title is now `chore(lockfile): refresh pnpm lockfiles`. - `check-lockfile-release-age.mjs` accepts more than one `--base`/`--head` pair. - `scripts/typecheck-test-programs.mjs`: if the frozen install of the provider workspace fails, the script resolves the lockfile in place and installs frozen again. - `scripts/build-standalone-public-packages.mjs`: the same fallback applies to a package with a committed lockfile (Daytona). `release.sh` already restores tracked files when it exits. - `doc/DEVELOPING.md` and `doc/RELEASE-AUTOMATION-SETUP.md` describe the managed lockfiles. They also say that the Pi and Kimi distribution lockfiles stay hand-managed, because they hold attested provider pins. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/__tests__/build-standalone-concurrency.test.mjs`: 731 tests pass. New tests cover: - the policy for provider lockfiles, including a real git history with a Dependabot commit and a later hand edit - the workflow and policy lists match - the unexpected-changes check, run with bash against a fixture - a held refresh round trip, and its invalidation when a provider lockfile changes on master - the release-age gate across lockfile pairs - the inline fallback in both install scripts - `node --test scripts/typecheck-test-programs.test.mjs`: the install tests pass. Two tests fail locally only because this worktree has no root `node_modules` (no `tsc`). They also fail without this change. - I ran the new bot commands on master with pnpm 9.15.4. No package version changed. pnpm 9.15.4 removes `libc:` fields and adds `deprecated:` notes, so the first refresh is a metadata-only change. The release-age gate passes it because it adds no versions. - I ran the release-age and held-lock steps locally with bash against this commit. The gate reported `eligible=true`, and `sha256sum --check` accepted the saved hashes. - The workflow YAML parses, and `bash -n` accepts every `run` block. - Not testable before merge: the policy job loads `lockfile-trust.mjs` from the base branch, so the new rule applies only to pull requests opened after this merges. The refresh workflow runs only on master. ## Risks - The release build now resolves a stale Daytona lockfile in place, from the registry. The root release install already resolves without a frozen lockfile, so this is the same level of trust. - The paid runner e2e job still installs Daytona with `--frozen-lockfile`. After a Daytona manifest change merges, its Daytona cases fail until the bot's refresh merges. This matches the rule that paid jobs refuse a new root lockfile until its refresh merges. - Dependabot pull requests for the provider entries in #19489 change a provider lockfile. They pass only if each lockfile matches a verified Dependabot commit, the same as for the root lockfile. - Low risk otherwise: the bot keeps its existing token, environment, and merge rules. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, and git). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:07:57 -08:00
DottaandClaude Opus 5.5	1485d3de1b	fix(ui): hold announcements until the onboarding first task is finished (#19514 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A new user's first session is the onboarding first task. The agent asks its onboarding questions in that task thread. > - The app also shows an optional in-app announcement. The live feed now serves "Introducing Connectors" (`2026-09-14-connectors`). > - The announcement well only waits while the onboarding wizard dialog is open. It shows 3 seconds after the user lands on the first task. > - In the Nov 5 cloud onboarding test (Claude API path), the tester said: "Don't show the connectors upsell until the second session. It's blocking the user flyout and potentially the onboarding questions." > - This pull request holds announcements until the company's onboarding first task is done or cancelled. > - The benefit is that a new user sees the onboarding questions and the account menu without an upsell on top of them. ## Linked Issues or Issue Description Refs #19511 (places the card in the desktop sidebar, above the account menu. That PR keeps the card off the account menu after onboarding. This PR adds the first-session gate that #19511 leaves open.) What happened? On a new company's onboarding first task, the "Introducing Connectors" card shows 3 seconds after the page loads. At 1440x900 the card (352x346 px, bottom-left, z-40) covers the sidebar account menu trigger and the left edge of the task composer. Expected behavior Optional announcements do not show in the first session, while the onboarding first task is open. Steps to reproduce 1. Create a new company through onboarding. The wizard creates the onboarding first task. 2. Open the first task at 1440x900 and wait 3 seconds. 3. See the announcement card on top of the account menu and the composer. Paperclip version or commit `master` at `819420a60a`. ## What Changed - `AnnouncementWell` reads the company's onboarding first task (`originKind=onboarding_first_task`, `limit=1`) with React Query under `["issues", companyId, "onboarding-first-task"]`. Issue live-update invalidation of `["issues", companyId]` refreshes it. - The well stays disabled while that task is not `done` or `cancelled`. While the read is loading or after it fails, the optional card stays hidden. A company that never had an onboarding first task is not gated. - "First session" uses the existing `origin_kind` marker. There is no schema change. - New e2e spec `tests/e2e/announcement-first-session.spec.ts` with a mocked feed that copies the live connectors entry. It checks that the card is absent while the onboarding first task is open. Then it marks the task done on the same page and checks that the card appears without a reload (a positive control for the feed mock and the live refresh path). - New unit test `AnnouncementWell.onboarding.test.tsx`. It covers open, finished, no first task, read failure, and the open-to-done transition after `["issues", companyId]` is invalidated. `AnnouncementWell.test.tsx` now gives the well a `QueryClient` with a finished onboarding state. ## Verification Red on `master` (`819420a60a`), before the fix. Vitest: ``` cd ui && npx vitest run src/components/AnnouncementWell.onboarding.test.tsx × stays hidden while the onboarding first task is open × stays hidden when onboarding state cannot be read Tests 2 failed \| 1 passed (3) AssertionError: expected true to be false ``` Red on `master`, Playwright at 1440x900 (an earlier draft of the spec also had a placement check, recorded here as evidence for the flyout part of the report): ``` ✘ announcement-first-session.spec.ts › a brand-new company with its onboarding task open does not show the upsell Error: expect(locator).toHaveCount(expected) failed Locator: getByRole('complementary', { name: 'Paperclip announcements' }) Expected: 0 Received: 1 ✘ … where the upsell shows, it does not cover the account menu or its flyout Error: well {"x":12,"y":542,"width":352,"height":346} covers account trigger {"x":20,"y":856,"width":166,"height":36} ``` Green after the fix, merged with current `master` (`f7070d68b5`), with the in-page positive control: ``` ✓ announcement-first-session.spec.ts › a brand-new company with its onboarding task open does not show the upsell (10.1s) 1 passed cd ui && npx vitest run src/components/AnnouncementWell.onboarding.test.tsx src/components/AnnouncementWell.test.tsx Tests 6 passed (6) cd ui && npx vitest run src/components/AnnouncementWell.onboarding.test.tsx src/components/AnnouncementWell.test.tsx \ src/components/AnnouncementCard.test.tsx src/hooks/useAnnouncement.test.tsx src/components/SidebarAccountMenu.test.tsx src/components/Layout.test.tsx Test Files 6 passed (6) Tests 95 passed (95) ``` - `pnpm check:token-gates`: All gates clean. - UI `tsc --noEmit`: no errors in app code. The only errors are the git-ignored, generated Storybook catalog files. - CI note: the first CI run had three jobs fail with "The self-hosted runner lost communication with the server" (server tests 12/12, Runner vitest package, e2e shard 5/8). These are infra flakes. I reran them. - Local e2e note: two runs could not start the server because embedded PostgreSQL ports were in use by parallel e2e runs on the same machine. A rerun passed. This is not related to the change. Before (master): the card covers the account menu and the composer on the onboarding first task. ![before: onboarding first task](https://raw.githubusercontent.com/paperclipai/paperclip/c97519868225b0c206cf95fa3fb7150287c88116/n13-before-onboarding-task.png) After: no card on the onboarding first task. The account menu and the composer are clear. ![after: onboarding first task](https://raw.githubusercontent.com/paperclipai/paperclip/c97519868225b0c206cf95fa3fb7150287c88116/n13-after-onboarding-task.png) ## Risks - Low risk. The change only hides optional content. - A user who never finishes or cancels the onboarding first task does not see announcements in that company. This is the intended trade-off for "not in the first session". Closing the task shows them again. - The well makes one extra small issues read (`limit=1`) per company. It is cached for 60 seconds. - `AnnouncementWell.tsx` also changes in #19511. Whichever PR lands second resolves a small conflict in the `enabled` condition and the `<aside>` placement. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, Playwright, Vitest). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 12:00:50 -06:00
DottaandPaperclip	5d25117ffa	fix(heartbeat): recover deferred dependency completion wakes (#19457 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks wait for other tasks through saved dependency edges and wake requests. > - A dependency can finish while its assignee answers a question about the blocked task. > - Paperclip saves the completion wake behind that active turn. > - Recovery did not find that saved wake unless it also contained a comment or an interaction response. > - This pull request restores that wake through the existing release and dispatch checks. > - The task can then continue after the active turn and its cleanup finish. ## Linked Issues or Issue Description What happened? A task stayed blocked after its dependency finished. The completion wake was saved while an audit turn was active. The wake stayed deferred after that turn ended. It had no comment or interaction response, so both recovery queries and the input guard excluded it. Expected behavior Recover the saved dependency wake when the prior turn and its cleanup finish. Start one dependency continuation under the current execution rules. Steps to reproduce 1. Run a question response on a dependency-blocked task. 2. Finish the dependency while the response is active. 3. Save the completion wake behind that response. 4. Finish the response and its environment cleanup, or restart the workers. 5. Observe that the saved wake has no run. Paperclip version or commit Reproduced against `70d23612dd`. Rechecked against `2f35eb1080` with the final regression fixture. Deployment mode Source checkout on Linux x64, Node 25.9.0, pnpm 9.15.4, and isolated embedded PostgreSQL. Related work: #16673 added recovery for saved messages. #17288 and #16966 address assignment wakes after cleanup. This change covers dependency completion receipts with no message. ## What Changed - Include saved `issue_blockers_resolved` receipts in worker reconstruction and stranded queue selection. - Let these receipts pass the input guard without a comment or interaction response. - Use the existing release transaction and dispatch checks. Add no timer, migration, or direct task status change. - Add process-backed regression cases for restart, cleanup release, cancellation, agent pause, pending cleanup, and unresolved blockers. - Update the dependency wake recovery document. - Share the dependency-wake payload predicate and SQL fragment across reconstruction and stranded receipt admission, so these eligibility checks have one owner. ## Verification - The final regression fails on the base code: the receipt stays `deferred_issue_execution` and has no successor. The same test passes with the fix. - Six focused cases pass. They use real PostgreSQL and a real process adapter. They check saved output, one dependency continuation, clean task locks, repeat recovery, and execution gates. They do not call a model provider. - Three nearby suites pass: 125 tests across queue wiring, dependency scheduling, and dependency recovery. - Final-head server layered typechecking passes with `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`. Whitespace and module boundary checks pass. Two parallel checker attempts reached the sandbox memory limit; the completed low-memory check covers all source and test layers. - The full deferred-receipt suite passes: 45 tests. Combined with the nearby suites, 170 local tests pass. These results cover head `8f9d3aa6ac19a6d2532bd869eb8adda7cf122a5a`. Required CI remains in progress. - Local workspace typechecking and build stop at the Runner Rust step because Cargo is absent. The full local test suite was not run. Required CI must supply broad coverage. - The hosted incident has not been recovered or deployed from this branch. - Review follow-up at `182385e9bc433adac06440997de7e4cfbf889a72`: normal merge of master `0d03610d2f73b7cd799547e43929bb2303319024`; no conflict or history rewrite. All 54 receipt/helper cases pass on macOS arm64, Node 26.4.0, real embedded PostgreSQL and process adapters. The six dependency continuation and safety cases also pass separately. Module boundaries and whitespace checks pass. No assertion, timeout, execution gate or retry policy changed for this extraction. - Current shared, SDK and adapter-utils source builds pass. The first broad local server typecheck read stale borrowed adapter declarations; rebuilding them exposed nominal type duplication between the borrowed workspace and this checkout. It is not counted as a server typecheck pass. All 30 SDK and 169 adapter runtime JavaScript outputs match the previously tested artifacts. Final revision `182385e9bc433adac06440997de7e4cfbf889a72` now passes all 55 applicable CI checks (three skipped, one neutral review), including clean-workspace typecheck, build and broad tests. [CI result](https://github.com/paperclipai/paperclip/actions/runs/42262173960). Greptile and Paperclip review both give 5/5 with no open findings. ## Risks - Recovery now considers a saved dependency receipt with no message. Current ownership, source finalization, cleanup, pause, cancellation, and dependency checks still control execution. - The scan remains bounded and uses existing transaction notifications and restart reconstruction. The integration checks verify that it becomes quiet after settlement. ## Model Used - OpenAI Codex, based on GPT-6. The exact provider model ID and context size are not exposed in this session. Used reasoning, tool calls, TypeScript edits, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 11:49:38 -06:00
DottaandPaperclip	e3fb4bc73a	fix(native): recover saved listener startup failures (#19479 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Native recovery must preserve saved work and respect current task authority. > - A closed local API listener can stop startup before a provider starts. > - #19462 gives new failures a bounded bootstrap retry. > - Older saved failures still have a generic terminal code and a recovery hold. > - This pull request verifies the old producer outcome and uses the existing safe replacement path. > - Eligible tasks continue once. Unknown provider outcomes and user holds remain blocked. ## Linked Issues or Issue Description Refs: #19462 What happened? A restart can leave an old listener startup failure in `terminal_failure` with `native_session_interrupted`. Startup recovery does not claim it after #19462. A completed file recovery can also retain its no-replay hold. Expected behavior Continue automatically when durable records prove that no provider started, the source stopped, and the current files were saved. Keep the existing attempt budget and normal admission checks. Steps to reproduce 1. Close the actual API listener after a fresh native coordinator claim. 2. Persist the old executor's generic failure. Save the workspace and confirm source termination. 3. Restart recovery. The baseline schedules no successor. The repair schedules one successor after all receipts pass. Paperclip version or commit Baseline: `0252421d5c`. Old failure producer: `2f35eb1080`. Deployment mode Built from source. Local macOS arm64, Node 26.4.0, real embedded PostgreSQL, independent filesystem execution. No external AI service is used by the controlled acceptance. ## What Changed - Recognize the exact saved local-listener guard through one proof predicate shared by discovery and the locked transaction. - Require a fresh first attempt, matching execution bindings, no prior provider lineage, no provider events or tool calls, no checkpoint or admission receipt, and no cancellation. - Reuse current snapshot, exact stop, source block, attempt budget, and successor idempotency checks. - Wait for completed source file-recovery receipts. Retire only that same source's verified preservation hold in the successor transaction. - Bound discovery to 100 candidates per pass, keep completed history quiet, and wake through existing transactional producer notifications. - Retain retries after transient database and final ownership-proof errors; advance the bounded cursor and force a rescan so a failing page cannot hide later work. - Let only the replacement duty discharge a proved-empty notification during drain through its existing read-only readiness check, once per notification generation. Keep real work and concurrent changes outstanding without running replacement during drain. - Wake replacement at canonical local-stop proof writes and the existing dispatch eligibility boundary; these hints grant no new execution authority. - Add database regressions, scheduler quietness checks, real controller handoff coverage, and an opt-in controlled Runner continuation. ## Verification Final production head: `0df41b2fef2644195d1224d36984af3827da3e3b`. Local environment: macOS arm64, Node 26.4.0, TypeScript 7.0.2, Vitest 5.0.3, frozen-lockfile dependencies. Tests execute freshly; `--fsModuleCache` reuses only content-validated module transforms. - Native safe replacement: 62/62 PASS on isolated PostgreSQL 17 (63.18 seconds), including clean per-suite database disposal. All 62 assertions also passed on embedded PostgreSQL 18 beta, but its Darwin cleanup hook timed out; that invocation is not counted as a pass. Docker's disk was full, so the disposable PostgreSQL 17 data used a size-limited RAM mount, with no host mounts. The owned container was removed afterward. - Actual stopped-process producer suite: 31/31 PASS. A real exited child produces the stop receipt, and its replacement notification settles only after the outer transaction commits. - Workspace and separate-process drain regressions: 33/33 PASS (75.90 seconds). - Shared coordinator and execution-control lifecycle checks: 39/39 PASS (16.43 seconds). Pending work, concurrent changes, cold startup, shutdown, and passive drain boundaries remain covered. - Actual suppressed Cloud routine/drain workflow: PASS (34.64 seconds), using the unchanged test and timeouts. Its matching local RED reproduced CI's 462-query, zero-pending-query, one-idle-hold timeout after an admitted write committed during drain (69.46 seconds). - Actual historical controller handoff: PASS (48.40 seconds). A separate controller loads the original executor from `2f35eb1080`, writes its generic listener failure, and exits. Recovery at the final head admits exactly one successor. The real Rust Runner and controlled Codex process complete governed `paperclip_finish`; attempt two commits and the saved file remains intact. Only the child loads the old artifact, and both isolated artifacts are restored afterward. Rust source subtree `77961b70f177ec3c38d11abcaec92fd3d0e96ec0` matches the built binary. No external AI service is used. - Seven corrected existing fixtures pass: transactional stop-proof calls (5 cases) and notification inventory (2 cases). Their earlier CI failures exposed fixture calls outside the documented transaction and an outdated expected queue set. - Server typecheck: PASS. Red/green proofs also cover the original saved failure on baseline `0252421d5c` (zero successors instead of one), held-history quietness, bounded forward scanning through 100 parked rows and concurrent wakes, real snapshot/cleanup producer notifications, actual issue-row lock timeout, and task/history eligibility notifications. An obsolete prior-run question can be skipped to request another proof check; actual skip while the task is paused or the source owns provider tool activity schedules no successor. The failed-page regression starts with 25 persistent verifier errors and a healthy candidate 26. Baseline RED repeats the first page. Repair GREEN advances the healthy candidate, revisits the original failures after their fault clears, creates all 26 successors exactly once, and becomes quiet. Greptile is SUCCESS, 5/5 on the exact final head (2026-11-06 17:09:21 UTC). Final-head CI is GREEN: 56 successful checks, three intentional skips, one neutral result, and no pending or failing checks. [Attempt two and results](https://github.com/paperclipai/paperclip/actions/runs/42258352876/attempts/2) include passing build, typecheck, Rust, native server, package, browser E2E, serialized server, general server/chat tests, and final verification. Five jobs in the first attempt were interrupted together at 17:13:12 UTC; the native-server log records an explicit runner shutdown signal. The single infrastructure retry reran only those five jobs and dependent verification, preserving the 49 successful checks on the unchanged head. No timeout was weakened, and setup failures or interrupted invocations are not counted as passes. Focused commands from the repository root: ```sh ./server/node_modules/.bin/vitest run --project @paperclipai/server server/src/services/remote-workspaces.integration.test.ts server/src/services/native-runtime/native-runner-restart-preparation.integration.test.ts -t 'native safe-replacement decision\|reconciles the drained remote backend' --fsModuleCache --reporter=dot ./server/node_modules/.bin/vitest run --project @paperclipai/server server/src/__tests__/execution-control-work.test.ts server/src/__tests__/delivery-work-coordinator.test.ts -t 'execution-control lifecycle\|work coordinator' --fsModuleCache --reporter=dot node node_modules/typescript/bin/tsc -p server/tsconfig.json --noEmit --pretty false ``` The historical acceptance sets `PAPERCLIP_TEST_HISTORICAL_LISTENER=1` and uses an original-executor child artifact with a `try/finally` restore. Normal regressions require no Rust binary. ## Risks - Compatibility is deliberately limited to positively proved fresh remote startup failures. Other old ambiguous failures remain held. - Existing provider work, a changed task block, incomplete preservation, or revoked authority must prevent execution. Normal retry admission remains in control. - This PR does not deploy the change or mutate a hosted task. Hosted recovery still requires the repaired revision and all current proof gates. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact backend model identifier and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 11:49:28 -06:00
DottaandClaude Opus 5.5	30a0b5982e	fix(runner): let Codex agents generate images with their existing provider key (#19516 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip runner launches Codex agents in an isolated app-server with a sandboxed shell > - On the Codex API path, the agent's OpenAI API key reaches only the trusted app-server process, never the shell > - The runner launched Codex with `--disable image_generation`, so Codex's built-in image tool was not available > - Codex then falls back to its `image_gen.py` script, which needs `OPENAI_API_KEY` in the shell, so the agent asked the user for a second key > - This pull request keeps Codex's built-in image tool on, so it uses the agent's existing provider auth, and lets the sandbox read only the folder where Codex saves the images > - The benefit is that a Codex agent can make images without a second key, and the provider key still does not reach the shell ## Linked Issues or Issue Description What happened? Manual onboarding test on Paperclip Cloud staging (Codex API path): "Open AI requests another api key for image model even though it has an active api key powering the agent." Steps to reproduce 1. Create a company with a Codex agent that uses an OpenAI API key. 2. Ask the agent to generate an image. 3. The agent asks for another OpenAI API key. Expected behavior The agent uses the OpenAI API key that already powers it. Root cause `createIsolatedCodexAppServerArgs` passed `--disable image_generation` and the thread config set `features.image_generation: false`. The pinned Codex 0.160.0 binary has two image paths. Its own instructions say the built-in `image_gen` tool "Does not require `OPENAI_API_KEY`". The fallback CLI path "Requires `OPENAI_API_KEY`". The runner removes `OPENAI_API_KEY` from the shell on purpose (`codex-shell-environment.ts`), so the fallback can never work and the agent asks the user. ## What Changed - `codex-security-config.ts`: remove `--disable image_generation` and `features.image_generation: false`. Codex's built-in tool runs in the trusted app-server with the agent's existing auth. - `codex-security-config.ts`: in the workspace and plan permission profiles, grant read on `CODEX_HOME/generated_images` only. Codex saves images there and tells the agent to copy them into the workspace. The rest of `CODEX_HOME` (for example `auth.json`) stays denied. Preconfigured-home mode is unchanged (it does not deny `CODEX_HOME`). - `eval-gateway-runtime.ts`: set `image_generation = false` in the controller-gateway Codex config. That gateway admits no hosted tools (`unqualified_hosted_tool`), so qualified evals keep the earlier behavior. - Tests: two new N-12 tests, and updates to the tests that asserted the old disable flag. ## Verification Red on `master` (`819420a60a`), same test file before the source change: ``` × N-12: lets an API-key Codex agent generate images with its existing provider key instead of asking for another 5ms × N-12: the pinned Codex binary keeps built-in image generation enabled under the isolated launch 4400ms ⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯ FAIL src/drivers/codex/codex-security-config.test.ts > Codex security configuration > N-12: lets an API-key Codex agent generate images with its existing provider key instead of asking for another AssertionError: expected [ '-c', …(22) ] to not include 'image_generation' 378\| expect(args).not.toContain("image_generation"); FAIL src/drivers/codex/codex-security-config.test.ts > Codex security configuration > N-12: the pinned Codex binary keeps built-in image generation enabled under the isolated launch AssertionError: expected 'image_generation …' to match /\strue\s$/ + Received: "image_generation stable false" Tests 2 failed \| 17 skipped (19) ``` Green after the fix: ``` Tests 2 passed \| 17 skipped (19) ``` Neighbouring suites (`src/drivers/codex`, `src/cli/eval-gateway-runtime.test.ts`, `src/provider-events.test.ts`): ``` Test Files 25 passed (25) Tests 429 passed (429) ``` Real sandbox check with the pinned Codex 0.160.0 binary (`codex sandbox` with the generated permission profile, macOS seatbelt): ``` generated_images/t1/img.png status=0 "PNGDATA" # agent can read the saved image auth.json status=1 "cat: ...: Operation not permitted" # rest of CODEX_HOME stays denied cp status=0 # agent can copy the image into the workspace ``` - `pnpm exec tsc --noEmit -p packages/paperclip-runner`: no errors in changed files (only pre-existing errors from the unbuilt `@paperclipai/paperclip-eval-kernel` package). - `pnpm verify` exits 2 (changes outside resolved test graphs); required CI supplies the broad coverage. - Test scope.* The red/green tests assert the launch configuration: the disable flag, the thread config, the sandbox grant, and the effective `image_generation` state that the pinned Codex binary reports. They do not drive a model. So they do not directly observe the reported "asks for a second key" behavior. The link from the flag to that behavior comes from the pinned Codex 0.160.0 bundled instructions: the built-in `image_gen` tool "Does not require `OPENAI_API_KEY`", and the fallback `scripts/image_gen.py` CLI "Requires `OPENAI_API_KEY`". The runner removes `OPENAI_API_KEY` from the shell. - Not run: a live paid image generation with an API-key Codex agent. There is no Codex staging company. ## Risks - Image generation uses the agent's OpenAI account. Images cost money on API-key billing. This is the expected behavior when the user asks for an image. - The agent can read images it (or an earlier thread of the same Codex home) generated. It cannot read other files in `CODEX_HOME`. - Codex's built-in `view_image` tool was already enabled, so the trusted process already reads image files for the model. This change does not add a new kind of file access to the provider. - If a future Codex release changes the image save folder, the agent can still see the image inline but cannot copy it. The N-12 binary test pins the feature state for the current Codex version. ### Security policy change (needs maintainer sign-off) Why it was disabled. #12364 ("isolate Codex security configuration") disabled host apps, plugins, memories, multi-agent behavior, and image generation as one group. Its goal was a closed launch configuration that does not depend on ambient Codex defaults. The PR records no specific threat for image generation. This PR keeps the other features disabled. Why re-enabling it is safe. - The tool is a hosted tool. It runs at OpenAI inside the same model request that the trusted app-server already makes. It adds no new network endpoint, process, or shell capability. - No credential moves. `OPENAI_API_KEY` stays out of the shell, argv, and serialized config (the N-12 test asserts this). - The new sandbox grant is read-only and covers only `CODEX_HOME/generated_images`. The rest of `CODEX_HOME` (`auth.json`, config, sessions) stays denied. The real `codex sandbox` check above confirms this. In plan mode the grant is also read-only. - Codex's `view_image` tool was already enabled. So the trusted process already sends local image content to the provider. - Controller-gateway evals keep the tool off (`image_generation = false`), because that gateway admits no hosted tools. Credentials and billing. - API-key path: the hosted tool uses the agent's existing OpenAI API key through the app-server. OpenAI bills image generation to that API account at image-model rates. - ChatGPT-subscription path: the tool uses the agent's existing ChatGPT sign-in. Use counts against that ChatGPT plan's limits. There is no separate API bill. - These two points follow from how Codex sends hosted tools in the provider request. I did not observe them in a live paid run. - Open question: Paperclip cost tracking uses provider token usage. I did not confirm that image-generation charges appear in Paperclip's per-agent cost and budget totals. If they do not, budget hard-stops could undercount image spend. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:48:54 -06:00
DottaandClaude Opus 5.5	19239c11d1	fix(onboarding): move Use a custom gateway under Advanced on the API key step (#19510 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - New users connect a model in the onboarding wizard's "Connect a model" step. > - On the Claude API and Codex API paths, the step shows an API key card. > - The "Use a custom gateway" button sat directly under the key field, inside that card, with no gap. > - A tester reported that it hugs the field and that it belongs under Advanced. > - A custom gateway is an alternative route, not the next step after the key. > - This pull request moves the button into the step's existing Advanced disclosure. > - The benefit is a cleaner key card, and the gateway route stays one click away. ## Linked Issues or Issue Description No public issue exists. A manual new-user onboarding test on a staging cloud workspace found this bug. Related: #17253 added the custom gateway route. What happened? On the onboarding "Connect a model" step, API key path (Claude API and Codex/OpenAI API), the "Use a custom gateway" button is shown at the top level, directly against the API key text field. Steps to reproduce 1. Start onboarding and reach "Connect a model". 2. Choose "Use API key instead", then pick Claude (or OpenAI). 3. See "Use a custom gateway" directly under the "Enter API key here" field. Expected behavior The custom gateway control is in the Advanced section of the step. It is hidden until Advanced is opened. Source: manual new-user onboarding test on Paperclip Cloud staging (item N-16, P3, Claude API path). ## What Changed - `ui/src/components/OnboardingWizard.tsx`: render `OnboardingGatewaySetup` inside the connect step's existing `Advanced` `<details>` (above the runner select), not inside the API key card. It keeps the same gates as the card it left: the card is live and mounted, no connect progress, API credential mode, a created company, and a Claude or Codex adapter. - `ui/src/components/onboarding/OnboardingGatewaySetup.tsx`: accept an optional `className`. The wizard passes `-ml-3` so the ghost button's label lines up with the Advanced summary. - The gateway dialog and its save/select-model/connect flow are unchanged. - `ui/src/components/OnboardingWizard.test.tsx`: new test for the Claude API and Codex API paths. It asserts the button is in a closed `Advanced` disclosure, outside the key field's card, and that the gateway dialog opens after Advanced is expanded. ## Verification Red on unchanged source (`npx vitest run src/components/OnboardingWizard.test.tsx -t "custom gateway"`): ``` ❯ src/components/OnboardingWizard.test.tsx (212 tests \| 2 failed \| 208 skipped) × keeps Use a custom gateway in the closed Advanced disclosure on the Claude API key step × keeps Use a custom gateway in the closed Advanced disclosure on the Codex API key step AssertionError: Use a custom gateway should live inside the Advanced disclosure: expected null to be truthy Test Files 1 failed (1) Tests 2 failed \| 2 passed \| 208 skipped (212) ``` Green after the fix (same command): ``` Test Files 1 passed (1) Tests 4 passed \| 208 skipped (212) ``` Neighbouring suites (`OnboardingWizard.test.tsx`, `ai-connections/AiProviderSetup.test.tsx`, `components/onboarding/*`), including the existing end-to-end custom gateway save/test/hire test: ``` Test Files 8 passed (8) Tests 270 passed (270) ``` - `pnpm check:token-gates`: all gates clean. - `tsc --noEmit` for `ui`: no new errors. The only errors are the existing unbuilt `@paperclipai/plugin-sdk/ui` import in `PluginOrganizationSwitcher`. Screenshots (Storybook `Onboarding/Agent arc › Connect · Claude API key`): \| Before (master) \| After (Advanced closed) \| After (Advanced open) \| \| --- \| --- \| --- \| \| ![before](https://raw.githubusercontent.com/paperclipai/paperclip/1dae9c09f9a4070c66cd313dbb64baf1636c54d0/n16-before.jpg) \| ![after collapsed](https://raw.githubusercontent.com/paperclipai/paperclip/1dae9c09f9a4070c66cd313dbb64baf1636c54d0/n16-after-collapsed.jpg) \| ![after expanded](https://raw.githubusercontent.com/paperclipai/paperclip/1dae9c09f9a4070c66cd313dbb64baf1636c54d0/n16-after-expanded.jpg) \| The red "This item doesn't exist or was moved." line in the screenshots comes from the story's runner fixture. It is present on master too and is not related. ## Risks - Low risk. This is a layout move of one control. The gateway flow and its gates are unchanged. - The control is now one click deeper. This is the intended outcome of the report. - After a gateway is saved, "Use a different API key" stays in the card, so a user can still leave the gateway route without opening Advanced. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, browser for Storybook screenshots). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:42:21 -06:00
DottaandClaude Opus 5.5	63e550b338	fix(ui): keep the audit hash out of the tool approval card (#19509 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents call connected-app tools (for example GitHub `create_repository`) through the tool gateway, and a write call waits for a human approval card > - The pending approval card showed an "Arguments and audit hash" disclosure. The disclosure printed the 64-character SHA-256 of the reviewed arguments > - A new user in onboarding testing called this "a long hash string in the review approval card" that is confusing and unapproachable > - The hash identifies the exact arguments for audit. The approver does not need to read it to decide > - This pull request removes the raw hash from the card text and keeps it behind a "Copy audit hash" action > - The benefit is a plain approval card, with the audit value still available to people who need it ## Linked Issues or Issue Description What happened? On the pending tool-approval card, the "Arguments and audit hash" disclosure printed the raw SHA-256 of the reviewed arguments (for example `8ca5312877235a…`, 64 hex characters) under the arguments JSON. Expected behavior The approval card uses plain language. Internal identifiers do not show as card text. A person who needs the audit hash can still get it. Steps to reproduce 1. Let an agent call a write tool on a connected app (for example GitHub `create_repository`). 2. Open the pending "Approve tool action" card ("Review request"). 3. Expand "Arguments and audit hash". The raw 64-character hash shows under the arguments. Paperclip version or commit Current `master` (`819420a60a`) and the Paperclip Cloud staging build used for onboarding tests. ## What Changed - New `ToolActionAuditHashCopy` button (`ui/src/components/ToolActionAuditHashCopy.tsx`). It copies the arguments hash to the clipboard through the shared `useCopyAction` hook, and shows "Copied" or "Couldn’t copy". - `IssueThreadInteractionCard` (`RequestToolActionCard`): the disclosure is now "Arguments". The raw hash `<p>` is replaced by the copy button. - `TaskChatCompactInteractionCard`: the same change in both places that printed `toolAction.argumentsHash`. - If the clipboard is blocked, the button shows "Couldn’t copy" and the hash then shows as selectable text, so the audit value stays reachable (review feedback). - Tests: a new regression test renders the review flow with the real staging payload shape (GitHub `create_repository`, raw gallery tool id, 64-hex hash) and asserts that no `/[0-9a-f]{12,}/i` text shows before or after the disclosure opens, and that the copy action writes the hash. The existing long-proposal test now asserts that the hash is not in the disclosure text. `ToolActionAuditHashCopy.test.tsx` covers copy success and the blocked-clipboard fallback. The clipboard mock is restored after the test. ## Verification Red on `master` (new test only, before the fix): ``` FAIL src/components/task-chat/TaskChatInteractionCard.test.tsx > TaskChatInteractionCard > keeps raw tool ids and the audit hash out of the tool approval card text AssertionError: expected 'Gcreate_repository · Name: brainrot-s…' not to match /[0-9a-f]{12,}/i + Received: "Gcreate_repository · Name: brainrot-storefront-prototype · Private: trueArguments and audit hash{\"name\":\"brainrot-storefront-prototype\",\"private\":true}8ca5312877235a4c0f6e9b1d2a7c3e5f9081b2c4d6e8fa1b3c5d7e9f0a2b4c6dDeclineApprove & run" Test Files 1 failed (1) Tests 1 failed \| 32 skipped (33) ``` Green after the fix (same test): ``` Test Files 1 passed (1) Tests 1 passed \| 32 skipped (33) ``` Neighbouring suites (`ui/src/components/task-chat`, `IssueThreadInteractionCard.test.tsx`, `AttentionQueueRow.test.tsx`, `transcript/RunTranscriptView.test.tsx`), after rebase on current `master`: ``` Test Files 62 passed (62) Tests 1047 passed (1047) ``` After the review fixes (`ToolActionAuditHashCopy.test.tsx`, `task-chat`, `IssueThreadInteractionCard.test.tsx`): ``` Test Files 61 passed (61) Tests 1004 passed (1004) ``` - `pnpm check:token-gates`: all gates clean. - `tsc --noEmit` on `ui`: no errors in the changed files. - Visual check in Storybook story `Issue Thread Interactions / Tool Action Pending Write`, disclosure open: - Before: the summary reads "Arguments and audit hash" and a mono line `sha256:9f2c1a7be4d0c8a3` shows under the JSON. - After: the summary reads "Arguments". A small ghost "Copy audit hash" button shows under the JSON. No hash text shows. ## Risks - Low risk. UI copy and layout change on one card type only. The server payload, the approval decision, and the stored hash do not change. - A person who read the hash off the card must now use the copy button. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with tool use (shell, file edits, browser). Default context window and reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:39:44 -06:00
DottaandClaude Opus 5.5	a0cba70f18	fix(runner): continue restart-interrupted ACPX and OpenCode turns (#19501 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs keep their provider conversation when the control plane restarts > - A local Codex turn lost in a restart already continues once in the same conversation (#15297) > - ACPX (Claude and the other ACPX profiles) and OpenCode took a different path: the runner closed the lost turn as a provider failure > - The task then blocked with "Provider turn failed", and the user's queued follow-up stayed unread > - A restart must not need a user action when continuing is safe > - This pull request gives ACPX and OpenCode the same recoverable restart-loss cause and the same single continuation as Codex > - The benefit is that work interrupted by a controller restart finishes on its own ## Linked Issues or Issue Description Refs #15297 (Codex continuation), #19399 (restart-interrupted turn hook), #19471 (harness fix used for the live reruns). What happened? After a control-plane restart during an active native ACPX Claude turn, the task ended `blocked` with `native_provider_terminal_failed: Provider turn failed`. The queued follow-up comment was not consumed. No continuation started. The next run then failed with `native_session_cleanup_quarantined`. A native OpenCode run killed mid-turn ended the same way (`turn.failed` with `error: null`). Expected behavior When the old provider process is gone and the conversation can be restored, continue the task once in the same conversation. When that is not safe, record an honest restart cause instead of a generic provider failure. Steps to reproduce 1. Start a native ACPX Claude task on a local runner. 2. Submit a follow-up comment while the turn is active. 3. Restart the control plane while the turn is still running. 4. Run `pnpm test:e2e:runner -- --max-automatic-retries 0 --id everyday-workflows.runner-acpx-claude.local.recover-controller`. Root cause - `runner-core/src/acpx_provider_backend.rs` `restore_session_if_needed`: a saved `turn_active` state always closed the session. It recorded `turn.failed` with only a top-level `code: acpx_active_turn_recovery_closed` and no `error` object. It also emitted `run.terminal`. - The runnerd transport forwards only `error`, so the controller saw `error: null` and reported generic `provider_turn_failed`. - `runner-core/src/provider_backend.rs` gave the recoverable `provider_turn_lost_on_restore` cause only to `provider_label == "codex"`. OpenCode shares that backend. - `nativeRestartInterruptedTurnId` accepted only `codex_app_server`. The runtime continuation gate accepted only `provider.kind === "codex"`. - Two follow-on defects appeared once the continuation ran live: - The restored ACPX provider rejected the controller's result for the lost turn's open semantic call. That call never settled. - The restored provider's MCP request ids restart. The controller keys semantic calls by `hash(runId, callId)`. The continuation's first Paperclip tool call reused an earlier id, was treated as already answered, and hung. After the result was accepted, `runner_settlement_incomplete` failed the run. ## What Changed - ACPX runner: for an active turn lost to a runner restart, prove that the original provider lifetime lost its fence quorum. Then restore the same conversation and record `turn.failed` with `error: { code: "provider_turn_lost_on_restore", recoverable: true }`. Keep the run open for the controller's continuation. An accepted result, an unanswered runtime request, or an active goal keeps the old close behavior. So do an unproven lifetime fence and a failed restore. - ACPX runner fallback close: add a structured `error` (`acpx_active_turn_recovery_closed`, not recoverable, with an honest restart message), so the cause survives the transport. - ACPX runner: track open semantic calls durably. On restore, retire the lost turn's open calls. Acknowledge their controller results as `retired` and do not send them to the new provider. - ACPX sidecar: when the sidecar restores an existing session, prefix forwarded semantic `callId`s with a per-process namespace (`r<hex>:<id>`). Fresh sessions keep raw ids. - OpenCode runner: give OpenCode the same recoverable cause and open run as Codex (`restart_lost_turn_is_continuable`). Cursor Cloud stays excluded because its agent runs remotely. - Runner TS: `nativeRestartInterruptedTurnId` accepts `acpx_runtime` and `opencode_server`. The runtime continuation gate accepts `acpx` and `opencode`. All other Codex safety rules are unchanged: one continuation, admitted dead-runner restart only, no compaction, goal, semantic result, or pending runtime request. - Docs: `doc/execution-semantics.md` and `doc/run-log-events.md`. ## Verification Red, then green, at each layer: - Rust `tests/acpx_warm_attachment.rs` `restores_the_conversation_after_losing_an_active_turn_to_a_runner_restart`. Red: status `closed`. Then red on the lost call's result: "tool result does not match a pending provider call". Green after the fix. - Rust `tests/native_provider_backend.rs` `restart_lost_opencode_turn_records_a_recoverable_cause`, with the qualified OpenCode launch fixture. Red: `error` was `null`. Green after the fix. - Rust unit test `restart_lost_turns_continue_only_for_locally_restored_providers` (Codex, OpenCode yes; Cursor Cloud no). - TS `native-session-runtime.test.ts` "continues a restart-lost acpx_runtime / opencode_server turn once". Red: `NativeProviderTerminalFailure provider_turn_lost_on_restore`. Green: exactly one continuation, which does not resend the task. - TS `acpx-runtime-sidecar.test.ts`: a restored provider's call ids are namespaced and resolve. - Server `native-runner-restart-recovery.integration.test.ts`: a checkpointed `acpx_runtime` or `opencode_server` restart interruption is claimed as `resume_dead_runner`. Red without the driver gate. Green with it. - Two `native_provider_backend.rs` tests asserted the old close-on-restore result. They now assert the new contract. Their real invariant still holds: the delivered response is never replayed. Suites run locally on the final revision: - runner-core: `cargo test --no-fail-fast acpx`, `--test native_provider_backend` (38), `--test acpx_warm_attachment`, `--test codex_provider restart_lost` - runner TS: `native-session-runtime`, `runnerd-restart-continuation`, Codex driver recovery, contracts, and `acpx-runtime-sidecar` (all pass) - server: `native-runner-restart-recovery.integration.test.ts` (80 passed) - `pnpm --filter "@paperclipai/paperclip-runner..." build` (all generated-contract checks pass) Live runner E2E (local, real Claude via ACPX, macOS arm64): - `everyday-workflows.runner-acpx-claude.local.recover-controller`: PASS twice in a row on the final runner code. The restart crossed an active turn. The task finished `done`, the queued follow-up was consumed, the ZIP download was verified, and there were no failure codes. - `stock-harness.runner-acpx-claude.local.continuity-restart` regression: PASS. - Earlier failing reruns are kept as evidence of the follow-on defects: one harness `kill EPERM` while stopping the server (unrelated to this change), one hang on the colliding call id, and one `runner_settlement_incomplete`. ## Risks - A continuation can run more work in the restored conversation. Its prompt requires inspecting state before repeating an uncertain action, as for Codex. - The ACPX restore waits up to 5 s for the original provider lifetime fence before it falls back to the old close. - The ACPX provider state file gains two fields with defaults. Older saved states still load. A downgraded runner cannot read state that a newer runner wrote with these fields. - Restored ACPX sessions now forward prefixed call ids. Only fresh sessions keep raw ids. Live proof covers Claude. Codex-ACP, Pi, Cursor, Copilot, Grok, Kimi, and Antigravity share this path, but this PR proves them only by unit tests and code review. - OpenCode is covered by the runner fixture and gate tests. The live OpenCode restart-chaos cell was not rerun here. - `runner-core` `unconfirmed_suspension_state_becomes_recoverable_only_after_cleanup_and_suspend` is flaky when run in parallel. It also fails with the new path disabled, so it predates this change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use, code execution, and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:39:29 -06:00
Nicky LeachandClaude Opus 5.5	98ebeefbc9	feat(environments): let managed config declare plugin environments (#19508 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document > - #19490 added the plugin environment rows, their marker index and write floor, but nothing creates them yet > - The managed config still accepts only one sandbox entry, so a managed instance cannot get a plugin-provided environment without manual setup > - This pull request lets the managed config declare plugin entries beside the sandbox entry, and provisions one locked row for each at boot > - The benefit is that a managed instance gets its plugin environments with no manual setup, and the rows recover by themselves when their plugin restarts ## Linked Issues or Issue Description Refs #19490 (the plugin environment rows this pull request provisions) Subsystem affected Server environments: the managed-config parser, the managed environment bootstrap, and the environment routes' write floor. Problem or motivation A managed instance cannot get a plugin environment from its managed config. The `environments` section accepts one entry, and that entry always provisions the managed sandbox. Company settings cannot create a plugin environment either. So each managed instance that needs one requires manual setup. Proposed solution Let an `environments` entry declare `driver: "plugin"`. Boot provisions one locked row for each plugin entry through the row layer from #19490. A later pull request adds a one-time instance default for one plugin entry, and advertises the new managed-config capability. Alternatives considered Manual setup on each instance, and a new control-plane endpoint. Both need a step outside the deploy. The managed config already delivers the sandbox entry. Roadmap alignment This extends the existing managed-config `environments` section, which already provisions the managed sandbox. It adds no new product surface. ## What Changed - `server/src/services/managed-config.ts`: an `environments` entry may set `driver` to `"sandbox"` or `"plugin"`. An entry without `driver` is still the sandbox entry, so today's documents parse unchanged. - A plugin entry has `name`, optional `description`, `pluginKey` (the plugin's manifest id), `driverKey` and optional `driverConfig`. - At most one sandbox entry. Names are unique across the section, and `pluginKey`–`driverKey` pairs are unique. Secret-like keys in `driverConfig` refuse startup, as for sandbox `config`. An unknown key or driver still refuses startup, and that includes `instanceDefault` until the follow-up pull request. - Plugin entries parse into a new `pluginEnvironments` list. The `environments` list keeps only the sandbox entry, so its consumers do not change. - `MANAGED_ENVIRONMENT_DRIVERS` lists the accepted drivers. - `server/src/services/managed-environments.ts` (`applyManagedEnvironments`): - Boot refuses to start unless an auto-installed catalog entry installs each plugin entry's `pluginKey`. `plugins.autoInstall` holds catalog keys, not manifest ids, so the parser cannot check this. Boot uses the manifest ids `createApp` already resolves from the bundled and distribution catalogs (`app.locals.managedBundledPluginKeys`, passed in by `server/src/index.ts`). Without that list it resolves the catalog itself. - Plugin entries run after the sandbox entry and reuse its readiness handling. Boot awaits `pluginsReady`, then requires the plugin `ready`, declaring the entry's driver, and running a worker. Otherwise boot archives the row and re-ensures it on the worker's `ready`. - Recovery re-ensures on every worker `ready` until one ensure succeeds or the plugin rejects the config (HTTP 422). - If the ensure fails because the worker died, boot archives the row and waits for `ready`. If the worker is still running (for example the call timed out), boot keeps the row as it is, retries once at once and on the next `ready`, and otherwise at the next boot. No timer is added. - Removing an entry never archives its row. - A plugin-only document does not conflict with `PAPERCLIP_EXECUTION_MODE`, because plugin rows do not use the sandbox slot. - Plugin entries never change the instance default. The sandbox stamp still fills only an unset or local default, so it never replaces a plugin default. - `server/src/routes/environments.ts`: `isPlatformSlotEnvironment` treats a marked plugin row as a live slot while the document declares its entry. So the marker-clear escape hatch is refused then, and allowed again after the entry leaves the document. - `doc/DEVELOPING.md`: new "Managed environments" section. ## Verification - `cd server && npx vitest run src/__tests__/managed-config.test.ts`: old documents parse unchanged. Sandbox plus plugin parses. Each of these refuses startup: two sandbox entries, `driver: null`, duplicate names, a duplicate pair, a secret-like key, an unknown key (including `instanceDefault`), an unknown driver, a malformed `driverKey` or `driverConfig`. - `npx vitest run src/services/managed-environments-plugin-entries.test.ts` (the sandbox-entry tests stay in `managed-environments.test.ts`; shared fixtures are in `managed-environments.test-support.ts`): - A plugin entry is ensured with the validator's worker manager, after the sandbox entry. - Boot refuses a plugin entry whose `pluginKey` no auto-installed catalog entry installs, both from the passed list and from its own catalog lookup. - A plugin that is down archives the row, and worker recovery re-ensures it. A missing plugin, a plugin without the driver, or a worker manager without `call` counts as unavailable. - Recovery stays armed after a failed attempt and stops after one success or a permanent config rejection (422). A `ready` during an attempt queues one more. - A worker that dies during the boot ensure archives the row and arms recovery. A failure while the worker keeps running leaves the row as it is and retries. - Removing an entry does not archive. Plugin entries and the sandbox stamp never change a plugin default. - An ensure failure is counted, not thrown. - `npx vitest run src/__tests__/managed-plugin-environments.test.ts` (embedded Postgres, real `applyManagedEnvironments`): the first boot creates the row and the second changes nothing. A plugin that is down archives the row, and recovery brings it back. Removing the entry does not archive the row. The instance default stays unset. The row tests from #19490 still pass. - `npx vitest run src/__tests__/environment-routes.test.ts`: the marker clear is refused while the document declares the entry, and allowed after it leaves. - `npx vitest run src/__tests__/instance-settings-service.test.ts src/__tests__/instance-settings-cloud-defaults.test.ts`: pass with the new `pluginEnvironments` field. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - No schema change. - A managed-config document with a plugin entry now provisions a row at boot. A document without one behaves as before. - Adoption renames an unmarked row for the same plugin and driver to the declared name and resets its driver config to the declared one. The row keeps its id, agents and other metadata, and it is then locked like the managed sandbox. - Boot now refuses to start when a plugin entry names a plugin that no auto-installed catalog entry provides. This is the same fail-closed posture as an unknown sandbox provider. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 17:35:13 +00:00
DottaandClaude Opus 5.5	543aa4d331	fix(tasks): show a pending native same-run resume instead of a competing Retry (#19503 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When a native run cannot reattach to its provider after a control-plane restart, the native coordinator resumes the same run by itself after about 30 seconds, up to three attempts > - The run projection shows this correctly as "Retry scheduled" with only `inspect_run` permitted > - The task interruption shown on the issue read only system-recovery evidence and `scheduledRetryAt`. It never read the coordinator's `nextAttemptAt`, so it showed `state: failed, canRetry: true` and offered a manual Retry while the automatic resume was still pending > - The two surfaces disagree, and a user click can race the automatic resume. That is one way a Retry gets parked behind "execution checks" > - This pull request makes the task interruption read the coordinator's pending resume time > - The benefit is one consistent story: while the automatic resume is pending, the task shows it with its time and no competing Retry. If recovery ends, exactly one Retry returns ## Linked Issues or Issue Description Refs #19487, #19497, #19426. What happened? A native run failed with `native_session_interrupted` after a restart. Its finalization row had `phase: retryable_failure` and `nextAttemptAt` 30 seconds later. The run projection said "Retry scheduled". The issue interruption said `state: failed, canRetry: true, nextRetryAt: null`. What did you expect? While the automatic resume is pending, the task shows that it is resuming, with the time, and no Retry. When the resume fails for good or never runs, exactly one Retry that works on the first click. Steps to reproduce 1. Run a native task and restart the control plane so the native attach fails. 2. Read `GET /api/issues/:id` while the coordinator's resume is pending. 3. Observe `interruption.canRetry: true` while `execution.phase` is `retry_scheduled`. ## What Changed - `task-interruption-presentation.ts`: the batched run read selects the native coordinator's `next_attempt_at` when the native run failed, has no result, and its finalization phase is `retryable_failure`. - A pending resume in the future shows `retry_scheduled` with its time and `canRetry: false`. - A due resume shows `working` for up to two minutes while its dispatch runs. After that, or once recovery is terminal, the task falls through to the existing single Retry. ## Verification - `task-interruption-presentation.test.ts`: new case for a future resume, a due resume, a stale resume, and no resume. It fails before the change (the task offered Retry) and passes after. - `task-interruption-query.test.ts` (embedded Postgres): the real batched query shows the pending resume for both list and detail reads, and returns one Retry after the coordinator becomes `terminal_failure`. - All 40 presentation tests and 17 query tests pass. Server typecheck passes. ## Risks - Low risk. This changes only the read model for tasks whose native coordinator holds a pending same-run resume. - If a resume never dispatches, Retry is hidden for up to two minutes after its due time before it returns. - The `POST /issues/:id/retry` endpoint is unchanged. A client that ignores `canRetry` can still submit a Retry during the resume. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file editing, embedded Postgres tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:33:46 -06:00
Nicky LeachandClaude Opus 5.5	ff1d1dea5d	fix(tools): reject non-tool connections before any write (#19492 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Apps and tools connect to Paperclip through tool connections > - `POST /api/companies/:companyId/tools/connections` creates a tool connection through `createConnection` > - The request schema accepts a channel purpose and the chat and voice transports, but `createConnection` cannot store them > - The database then rejects the row, so the caller gets a 500 and a stray tool application stays behind > - This pull request rejects those requests with a 422 before any write, and stores `config` as `transportConfig` when the caller leaves `transportConfig` out > - The benefit is a clear error, no leftover rows, and consistent connection rows ## Linked Issues or Issue Description No issue exists. The new test typecheck in #18739 found this problem. #18739 listed it as a possible product problem and did not change it. What happened? 1. `createConnection` never writes `input.connectionPurpose`, so every row gets the database default `tool`. 2. The schema allows `transport: "chat_sdk"` and `transport: "voice"`. The database check `tool_connections_channel_transport_check` does not allow these transports on a `tool` row. 3. No error mapping exists for a check violation, so the route returns 500. 4. When the request has no `applicationId`, the service inserts a `tool_applications` row first, outside the connection transaction. That row stays after the failure. 5. The schema gave `transportConfig` a default of `{}`. The service stores `input.transportConfig ?? config`, so it stored `{}` instead of a copy of `config`. Expected behavior The route returns 422 for a purpose or transport that it cannot create, and writes no rows. When the request leaves `transportConfig` out, the row stores `config` there. Steps to reproduce 1. Send `POST /api/companies/:companyId/tools/connections` with `{ "name": "x", "transport": "chat_sdk" }`. 2. See the 500 response. 3. See the new `tool_applications` row for the company. ## What Changed - `server/src/services/tool-access.ts`: `createConnection` rejects a purpose other than `tool`, and the `chat_sdk` and `voice` transports, with 422. The check runs before any lookup or write. Channel connections still come from their own services. - `packages/shared/src/validators/tool-access.ts`: `transportConfig` is optional and has no default. The update schema already removed defaults, so it does not change. - `server/src/__tests__/tool-access-service.test.ts`: added tests for the three rejected inputs (no rows written), and for a request that sends only `config`. ## Verification - `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/generic-mcp-connection.test.ts server/src/__tests__/tool-connection-health-work.test.ts server/src/__tests__/openapi-routes.test.ts server/src/__tests__/tool-access-routes.test.ts packages/shared/src/validators`: 27 files, 995 tests pass. - `cd server && pnpm typecheck`: passes. This includes the source layers and the three test-tree shards. - Not run locally: the full `pnpm test:run`, the root `pnpm build`, and the UI typecheck. Required CI covers them. ## Risks - Low risk. A client that sends a channel purpose or a chat or voice transport to this route now gets 422 instead of 500. The request failed before this change too. - A client that sends only `config` now gets the same value in `transportConfig`. Every reader checks `config` first, so this changes no behavior. - The OpenAPI output for this body no longer shows a default for `transportConfig`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 17:30:24 +00:00
DottaandClaude Opus 5.5	f5de197536	fix(ui): keep a busy-workspace wait as a quiet status, not a notice (#19507 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The task page tells the user what an agent is doing and what needs their attention > - When another task uses the same workspace, the agent waits and Paperclip starts the work automatically > - A tester reported that the "Waiting for workspace" notice is not helpful, because the user cannot do anything about it > - On `master`, the task page shows this wait four times: a pinned top banner, a conversation banner, the composer strip and the Workspace property row > - This pull request removes the two banners and keeps the two quiet status surfaces > - The benefit is that a transient system wait no longer looks like something that needs the user ## Linked Issues or Issue Description Refs #19369 What happened? During new-user onboarding tests on Paperclip Cloud staging (Claude subscription path), the tester saw a "Waiting for workspace" notice and reported: "this isn't helpful and the user can't do anything about it." Steps to reproduce 1. Start work on a task while another task uses the same workspace. 2. Open the waiting task. 3. See the pinned "Waiting for workspace" banner under the title and a second "Waiting for workspace" banner in the conversation. Expected behavior A busy workspace is a transient system wait with no user action. It shows only as a quiet status on the task. It does not show as a banner or notice. Investigation - No server inbox, Needs-you item or activity entry is created for a workspace wait. The live-updates toast already skips `workspace_busy` (`LiveUpdatesProvider.tsx`), and the chat thread already hides the cancelled admission attempt. - The remaining notices are UI banners: `IssueMonitorBanner` (pinned top banner) and `IssueScheduledRetryCard` (conversation banner). - #19369 stopped showing resource waits as failed attempts, but it kept these banners. ## What Changed - `IssueMonitorBanner`: render nothing for a workspace wait. The composer strip still shows "Waiting for workspace" and "You can keep sending instructions while the agent waits." - `IssueScheduledRetryCard`: render nothing for `workspace_busy`. The Workspace property row still shows "Waiting for workspace". - `DESIGN.md`: add the rule that a busy-workspace wait is a quiet status only, never a pinned banner, conversation notice or toast. - Tests: replace the old "shows the banner" assertions with N-14 regression tests. ## Verification Red on `master` (`819420a60a`), test change only: ``` cd ui && npx vitest run src/components/IssueMonitorBanner.test.tsx src/components/IssueScheduledRetryCard.test.tsx × does not raise a pinned banner for a workspace wait the user cannot act on (N-14) 7ms ⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯ FAIL src/components/IssueMonitorBanner.test.tsx > IssueMonitorBanner / IssueMonitorComposerStrip rendering > does not raise a pinned banner for a workspace wait the user cannot act on (N-14) AssertionError: expected 'Waiting for workspaceAnother task is …' to be '' // Object.is equality FAIL src/components/IssueScheduledRetryCard.test.tsx > IssueScheduledRetryCard > raises no conversation notice for workspace contention the user cannot act on (N-14) AssertionError: expected 'Waiting for workspaceAnother task is …' to be '' // Object.is equality Tests 2 failed \| 32 passed (34) ``` Green after the fix, same command: ``` Test Files 2 passed (2) Tests 34 passed (34) ``` Neighbouring suites: ``` cd ui && npx vitest run src/components/TaskChatThread.test.tsx src/pages/IssueDetail src/components/issue-properties src/lib/issue-monitor Test Files 7 passed (7) Tests 687 passed \| 1 skipped (688) ``` - `pnpm check:token-gates`: all gates clean. - `tsc --noEmit -p ui`: no errors in the changed files. Local errors come only from unbuilt workspace packages (`@paperclipai/plugin-sdk/ui`, Storybook feature map). CI typecheck covers the full build. ## Risks - Low risk. UI-only change. Two banners no longer render for `workspace_busy`. Other scheduled-retry reasons (failures, pool exhaustion, continuations) are unchanged. - A long workspace wait now has no prominent notice. No actionable next step exists in the current data (the API does not name the task that holds the workspace). A persist-past-threshold escalation with a link to that task can follow when the data exists. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edit, browser for read-only staging evidence). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 11:30:10 -06:00
DottaandPaperclip	0d03610d2f	fix(opencode): export bounded tool argument diagnostics (#18699 ) ## Thinking Path > - Paperclip manages work across agent runtimes. > - Company observability sends allowed records to the operator's OTLP receiver. > - Native OpenCode exports tool results but omits submitted argument diagnostics. > - Exact argument values are excluded by the current capture policy. > - Safe type information can distinguish malformed envelopes without copying private values. > - This change exports a fixed diagnostic shape from authenticated dispatch and schema-rejection events. > - Operators can compare rejected and accepted calls under the existing capture controls. ## Linked Issues or Issue Description What happened? OpenCode tool spans omit argument diagnostics. The exporter reads native tool events. The authenticated MCP boundary has the submitted arguments, but its dispatch item events do not create tool spans. Calls rejected by its schema check do not emit dispatch items. Expected behavior With tool details and tool content enabled, Paperclip tool spans include the method, diagnostic schema version and allowed argument field types. Exact values and arbitrary field names stay excluded. Steps to reproduce 1. Configure a company OTLP receiver and enable native OpenCode export. 2. Enable tool details and tool content. 3. Submit a rejected Paperclip tool call and then a valid call. 4. Inspect the input diagnostics and outcome of each span. Related work: #17989 proposes metadata-only native export. #18098 added the normalized content exporter. This change repairs argument diagnostics in that exporter. ## What Changed - Export `paperclip.opencode.tool-diagnostics.v1` as `braintrust.input` for authenticated Paperclip tool calls. - Include `tools/call`, the argument type and types for a fixed list of envelope fields. - Use dispatch IDs for Paperclip tool spans. Keep separately named `opencode tool ...` provider records for failures before dispatch. Mark each span source. - Preserve schema-rejected calls as failed items without invoking the controller handler. Keep authentication and the closed tool catalog. - Bound schema-rejection error text in stored events and transcripts with the existing redaction and 8 KiB limit. Keep the provider response unchanged. - Preserve the controller opt-in, tool-detail and tool-content switches, redaction, limits and run identity. - Add MCP-to-receiver regression coverage and update the capture policy and capability maps. ### Hypothesis and thesis for a fix The exporter drops the dispatch diagnostic signal. Raw input omission is intentional. The repair belongs in the OpenCode driver and exporter. Shared protocol, tool permissions and task completion stay unchanged. The regression sends schema-rejected, handler-rejected and valid argument envelopes through a fixture OpenCode subprocess and the authenticated MCP bridge. It checks the downstream rejection and receipt, completion event and actual local OTLP request. The baseline has no dispatch spans. The candidate has separate spans with safe type diagnostics. Two warm runs and disabled capture use the same path. Source: [OpenCode diagnostic pattern](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/f172172e-9c65-453f-bb3a-7c6ba4f5a400). ## Verification - Baseline: `a92f616cd0a457c8ba85d994b24e1f92c2e4d1de`. The new integration regression fails because tool spans are absent. The same regression passes with this change. - Exporter and MCP bridge suites: 37 tests pass. Three final driver controls pass. Ten selected controls also passed before the final provider-record repair. Runner TypeScript typecheck and build pass after the review repair. - The review regression failed with two spans before the repair. It passes with three spans after the repair. Unauthenticated and unassigned calls do not reach the rejection observer. A broken observer cannot change the schema rejection. - The full OpenCode/exporter/MCP run passes 94 tests and fails one existing final-answer selection test. The same failure occurs in an untouched baseline checkout. It is not claimed as a pass. - A further filtered run reached the default five-second timeout in two existing continuation tests during concurrent repository checks. All five selected dispatch, warm-session and continuation controls pass with a 15-second test deadline. The earlier timeouts remain recorded. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Runner TypeScript checks and build pass. Full repository typecheck and build stop at Rust because `cargo` is not installed. The full repository test command was stopped before workspace handoff. No full-suite pass is claimed. Local source commits: `c48a8af1c186e415c802fa0ac95209b16b4a3122` `6744991f6`, and `a8ccf4693`. The remote branch is based on newer master `4814d1ab691c176170510a93a175dafdc9e021c7`. All initial eight changed files match between those bases before the patch. - The integration provider and controller are fixtures. The OTLP receiver is a real local HTTP server. A separate real OpenCode 1.18.34 probe opened a session. The remote free model returned `Free usage limit reached`. No tool call or receipt was produced. No live recovery is claimed. - Review follow-up: a real MCP request with 12,000 invalid array elements produced a 264,908-character event error before the repair. The same regression now stores 8,192 characters in both events and transcript. The MCP error receipt stays unchanged and no handler runs. All 37 exporter/MCP tests and 12 selected driver/exporter/MCP controls pass. Runner TypeScript checks and build pass. Local follow-up commit: `64d4fa583`. - The previous remote head passed Runner, typecheck, build and E2E. Its unchanged attention-retention shard failed twice with 15-second timeouts, then cleanup deadlock and duplicate fixture IDs. Both affected retention suites passed all 70 tests locally unchanged. The new head needs fresh CI and review; no full CI pass is claimed. - Current-master qualification (2026-11-06, merge `2959ecdb8d`): normally merge master `53645a0104`, including the existing test-only retention-fixture repair from #18774 reused by #18770. Keep the OpenCode diagnostics feature and upstream durable MCP and completion-feedback behavior. No test deadline or assertion changed. The three old CI timeouts did not reproduce locally: the four completed-blocker-history cases passed on the old head in 29.52 seconds, then passed on the merged head in 13.23 seconds. The fixture rollback/provenance regression also passed. All 116 exporter, MCP bridge and OpenCode driver fixture tests passed. Runner TypeScript checks and both TypeScript compilation targets passed; ACPX profiles match the release manifest and protocol-contract checks pass. These checks use local PostgreSQL, HTTP and process fixtures. No hosted provider call or deployed recovery is claimed. Fresh CI and review are pending for this merge. - Map-scope review follow-up (`61e662e179`): remove only the inherited diagnostics paragraphs from the company capability maps. This change adds logging detail within an existing capability. `node scripts/check-maps.mjs` and `git diff --check` pass. Production and test inputs are unchanged from the qualified merge; fresh CI and review are pending for the final head. - Test compiler follow-up (`6675b46377`): reproduce CI TS2722 with `tsc -p packages/paperclip-runner/tsconfig.test.json --noEmit`. The harness contract makes transcript retrieval optional. The OpenCode bounds regression now fails explicitly if that capability is absent, then inspects the narrowed method. The same complete Runner test compiler project passes, and the actual local schema-rejection fixture passes with all receipt and 8 KiB assertions retained. Production inputs are unchanged. Fresh final-head CI and review are pending. - Final-head qualification: `6675b4637743aa269e5214756b158e1e0a4561a5` passes all 56 applicable CI checks (three skipped, one neutral review). [CI result](https://github.com/paperclipai/paperclip/actions/runs/42253119577). Greptile gives 5/5 with no open findings. Hosted task recovery remains pending verified rollout. ## Risks - Authenticated spans use dispatch IDs. Provider records use separate names and a source attribute. Consumers must choose the source they need to avoid counting two observations of the same tool action. - Safe types cannot reconstruct nested arguments or explain every value-level validation failure. - Export remains best effort and bounded. Lost native frames no longer remove dispatch diagnostics, but abrupt process termination can still lose a turn. - Full local Rust checks and a deployed product journey are not yet verified. The pull request is unmerged and not deployed. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use and code execution. The exact runtime variant and context-window size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1106.0-canary.17	2026-11-06 11:01:25 -06:00
DottaandPaperclip	952ecdbb39	feat(ui): show assignee avatars in task lists (#18607 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People use task lists to see who owns each task. > - The shared task table shows an assignee name without a picture. > - A picture helps people find an agent or person when they scan the list. > - This pull request shows the existing avatar beside the assignee name. > - The table uses the same image and fallback components as other identity views. ## Linked Issues or Issue Description What existing behavior does this improve? The Assignee column in the shared task table. Subsystem affected `ui/` — the task table and its page data. Current behavior The column shows only an assignee name. People must read each name to find an owner. Proposed behavior Show the agent avatar or human profile image beside the name when the Assignee column is visible. Use the existing fallback if an image is missing. Show no avatar for an unassigned task. Reason and benefit People can use the same identity pictures throughout the app. This helps them scan task ownership. Breaking changes None. The API, task assignment, name sorting, and filters keep their existing contracts. Additional context Related work: #2422 adds agent avatar URL support. It does not add avatars to this table. #13319 aligns icons in task properties. This change uses the existing identity components. ## What Changed - Pass agent appearance and human profile images from existing company data to the task table. - Keep the avatar visible beside a long name as the name scrolls on hover or focus. - Respect the Assignee column setting and keep unassigned rows without avatars. - Load human photos only from Paperclip asset URLs. Show initials for external profile URLs without an image request. - Add tests with the real task row for agent and human images, image fallbacks, unassigned tasks, and the Columns control. - Use the shared human label when a directory entry is missing. The local operator shows Board. Other missing users show the short user ID. - Update Storybook data, design guidance, and the task-list verification steps. ## Verification - Current head: `c49fee5c9b`. The source-tested merge is `586873fe47921b250ab0de18371d25632f46cff0`, which integrates master `53645a0104`. The last commit removes only the inherited avatar paragraph from the feature map under the current map-scope rule. Map checks pass again. No history rewrite or duplicate feature PR. - Reproduced the old voice-report CI failure before integration. The unchanged test on `7f1d137a` passed on an unloaded local machine. With a controlled 3,250 ms delay in its first local provider fixture and a 15-second wait, the old polling failed: expected 105 report requests, received 315. This confirms the fake clock crossed refresh deadlines during database work. - Master already fixes this exact failure through #18628 (`d26628e35a`). The fixed polling passes with the identical delay and wait budget. The exact session IDs, 105 requests, one attempt per report, bounded reconstruction, restart, and quiet assertions remain. The temporary delay is removed. - Final committed code: all 15 voice-report tests and 46 scheduler/coordinator tests pass against disposable embedded PostgreSQL on macOS arm64. No external provider call is used. - All 154 tests pass across eight focused UI suites: TaskList, TaskList.assignees, CoordinatorTasks, IssueRow, AgentAvatar, useTaskListCollection, useTaskListColumnPreferences, and TaskChatBubble. - Three merge conflicts retained both current behavior and the avatar feature: current sorting and column preferences, unread state, assignee photos, and shared human labels. The focused tests cover those contracts. - UI TypeScript, UI build, token gates, map structure, Storybook catalog generation, and whitespace checks pass. Storybook output is generated and ignored. - The original server shard failure remains recorded: [voice-report CI failure](https://github.com/paperclipai/paperclip/actions/runs/41472540522/job/125369899753), expected zero idle holds but received two. Older attempts also had attention-retention and sandbox-runtime timeouts; their causes were not established in those attempts. Fresh CI must qualify the merged head. - Previous review findings were fixed with red/green coverage: missing avatar tests, shared human labels, and arbitrary human image URLs. Current code still limits human photos to Paperclip asset URLs; external URLs use initials and start no image preload. - Prior browser checks covered agent and human images, fallbacks, unassigned rows, hidden columns, and long-name keyboard focus. Those checks used production components with fixture data and do not prove live backend behavior. No new browser or hosted task action was performed for this CI repair. - Current head `c49fee5c9b`: Greptile and Paperclip review both give 5/5 with no findings. The initial CI attempt passed the original server shard and all other completed test suites. The workspaces-a 2/2 job was interrupted by a runner shutdown signal ([job log](https://github.com/paperclipai/paperclip/actions/runs/42250845315/job/128006963628)); it has no complete test result. A single bounded retry of that shard and its dependent verification has been requested after the initial workflow ended. No source change is based on that interruption. - Final-head qualification: `c49fee5c9b75b695d2894933bbcbe0896aab9148` passes all 55 applicable CI checks (three skipped, one neutral review). [CI result](https://github.com/paperclipai/paperclip/actions/runs/42250845315). Greptile gives 5/5 with no open findings. Hosted task recovery remains pending verified rollout. ## Risks - Low risk. The change adds identity pictures through existing UI components and existing company data. - Human profile images hosted outside Paperclip show initials in this table. This matches the existing task-message avatar rule. - A long name has less width beside its avatar. The existing hover and focus reveal keeps the name readable. - Current master changes table sorting and preferences. Merge resolution retains these contracts and their tests. Fresh CI and review must pass before merge. ## Model Used OpenAI GPT-6 through Codex assisted with code and verification. The runtime did not expose an exact model ID or context-window size. The agent used code execution and repository tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — current head `c49fee5c9b`; the map-scope finding is fixed and its thread is resolved - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 11:01:02 -06:00
DottaandPaperclip	c6573962f8	fix(workspaces): recover full sandboxes through read-only export (#19465 ) ## Thinking Path > - Paperclip preserves agent work across failed runs and server restarts. > - A remote workspace must have a durable save before its source can be released. > - Portable capture normally writes a Git mirror and a full archive inside the sandbox. > - A full sandbox can prevent capture before the provider's streaming download starts. > - Retrying that write cannot create the required free space. > - This change transfers a checked standalone source and seals its archive on the controller. > - The source remains retained unless the existing durable save completes. ## Linked Issues or Issue Description Related: #18644 reduced temporary capture space. #18172 added the provider's disk-free directory transfer. This repair connects portable capture to that transfer when the archive itself cannot be created. What happened? Workspace capture returns `remote_workspace_sandbox_disk_full` after provider work has finished. File recovery repeats the same sandbox write. Expected behavior Recover a stable standalone workspace without requiring another full archive on the source disk. Preserve staged changes, untracked work, confined links and the original source. Steps to reproduce 1. Create a standalone Git repository with a commit, staged changes and an untracked result. 2. Use the existing real-command transport fixture. Inject ENOSPC into the sandbox helper write or archive write. 3. Run portable capture. Before this change it rejects the helper failure. 4. With the change, restore the saved archive after removing the fixture source. Check the commit, staged content, untracked result and Git connectivity. ## What Changed - Attempt one controller-side capture after an exact sandbox disk-full failure. - Read a source inventory without installing a helper. Use the existing provider bulk transfer with confined links and runtime exclusions. - Compare the transferred inventory with source fingerprints before and after sealing. - Reject copied Git pointers, alternates and administrative symlinks. Parse config as data without includes; allow only portable keys before controller Git normalization. - Preserve socket exclusions. Retain the source and remove incomplete controller staging after failure. - Update execution semantics and focused regression coverage. No database migration or new execution authority. ## Verification - TDD red on base `2f35eb1080`: the new full-sandbox test failed with `remote_workspace_sandbox_disk_full` before production edits. - Green: 202 transport and archive tests passed before the additional Git configuration guards. At head `8541086cb5`, all 116 transport tests and the PostgreSQL durability acceptance pass. The final `e24e173ec9` exclusion guard passes those 116 cases plus its new Git-history regression across the full run and focused rerun. Environment: macOS arm64, Node 26.4.0; real Node/Git commands and filesystem transfer. - Review regressions were also red before their fixes: an internal Git config symlink bypassed the check, and `core.attributesFile` could select a controller path. Git administrative symlinks and nonportable config now fail before normalization. The focused safety selection passes 11 cases. - Cases cover helper and archive ENOSPC, commit/staged/untracked continuity after source removal, confined links, sockets, changed source, corrupt or extra copied files, unsafe Git metadata, retained diagnostic text and failed download. - Real PostgreSQL acceptance passes: normal workspace save recovers one sandbox ENOSPC, commits the encrypted snapshot, clears its writer, loses the original allocation, then independently reloads the saved result twice. The first acceptance attempt used an incorrect expected status (`available`); the documented state is `committed`. No production change was needed for that correction. - Review regressions for untracked Python/Node environments and excessive Git roots failed before their fixes. Dependency exclusions now match normal capture. Controller parsing starts only after whole-tree/repository budgets and layout checks; refspec/settings budgets and a shared two-minute verification/normalization deadline bound further work. The excessive-root test verifies zero Git subprocesses. - Real provider-boundary red/green: Daytona direct tar and capacity-fallback PTY transfers first dropped `._result.txt`. The new optional mapping contract preserves all user names and excludes only current transfer scratch. Kubernetes implements the same contract. Explicit caller exclusions and archive validation remain in force. - Standalone Linux qualification on Node 24.22.2/Debian bookworm: fresh shared/SDK build; Daytona and Kubernetes typecheck and build; Daytona file-sync/capacity/diagnostics 68/68 pass; Kubernetes file-sync 31/31 pass, including the actual `/proc` case. Qualification was refreshed again on the final `be6f7feacc` source tree: current shared/SDK rebuilt; all 815 staged source/config/manifest hashes match the delivered tree before and after testing. An initial container run as root bypassed a read-only file test; the same source passes as uid 1000. Installing zstd enabled eight initially skipped cases. - A further review regression proved `.git/pyvenv.cfg` could exclude Git history as a dependency. The inventory now handles `.git` before dependency filtering, matching normal capture. The full capture/restore test verifies the original commit and saved work. The first green attempt incorrectly compared command-result object identity; correcting the test to compare stdout passed with no production change. - Further red/green regressions cover read-only private-copy cleanup, a regular file replaced by a FIFO before open, case aliases of Git administrative metadata, and permission preservation under restrictive umask. Nonblocking no-follow file handles verify identity before and after reading. Cleanup changes only private controller directories. Provider extraction preserves ordinary permissions, refuses special mode bits before writing, and does not adopt source ownership, ACLs, attributes or flags. Final macOS Daytona tests pass 65/65; the final Linux qualification above was repeated on these exact provider sources. - Focused strict TypeScript compilation of the inventory and archive modules passes. - The first sandboxed archive run failed three Unix socket fixtures with EPERM. The permitted run outside that filesystem sandbox passed all tests. - Initial full local server typecheck could not resolve the unbuilt Runner package and produced cascading errors. Exact-head `e24e173ec9` CI typecheck, build, Runner and browser checks pass. That CI attempt failed two server shards: the unchanged retained-cleanup backlog test timed out at 15 seconds and passes locally unchanged; ten cache-reclaim fixture oracles failed because directory fallback consumed their sealed-archive counters/faults. Final commit `caed4e4f83` isolates the intended unavailable-directory boundary while preserving archive counts, recovery status, integrity and no-second-reclamation assertions. Those ten failures reproduced locally before the fixture edit. The full affected file passes 394/395; its sole other HTTP socket-hang-up case passes unchanged in an isolated rerun. This is combined coverage of all 395 cases, not a single complete green run. The final branch integrates master `53645a0104` at `98ea07225592930cbb0d18bcf250bc64f6bb13b4`, preserving upstream workspace phase timings. After integration: 117 transport cases, the PostgreSQL durability acceptance, the existing object-storage/timing regression, and 12 affected cache cases pass. The expanded recovery file now has 421 cases. Final-head `be6f7feacc45c1e32a26a2417e973f1e3752fd8d` CI passes all 55 applicable checks (three skipped, one neutral review), including the expanded recovery suite, typecheck, builds and Runner coverage. [CI result](https://github.com/paperclipai/paperclip/actions/runs/42251345554). - Provider exclusion regressions were red through four real Linux transfer paths: Daytona direct archive, direct safe-link repair, capacity PTY repair, and Kubernetes streaming. A root socket and nested regular file share a basename; literal glob-character files/subtrees also survive. Preserve mode now anchors tar exclusions as literal root-relative paths and escapes `find -path` pruning. Default callers keep glob semantics. Receipt-count regressions were also red before correcting preserved-output counting. All 31 Kubernetes and 68 Daytona tests, builds and typechecks pass on the final source with 815 matching inputs. - The final semantics documentation explicitly records special mode bits, the stricter config allowlist (including innocuous unlisted keys), and provider qualification limited to Daytona/Kubernetes; these address the optional current-head review observations without broadening controller Git authority. - No Daytona credential is available in this session. Provider transfer is a local filesystem boundary in these tests. Hosted task recovery and rollout remain unverified. ## Risks - Recovery reads the source repeatedly and transfers the standalone tree. It can take longer than normal capture. - Linked worktrees, initialized submodules, Git administrative symlinks, nonportable Git configuration, special mode bits and alternate object stores remain retained if the normal capture cannot finish. This fallback does not follow their pointers onto the controller. - A provider without usable directory export, a changed source, or insufficient controller space cannot produce a receipt. The original source stays retained. - The server, SDK and affected provider plugins must be built from this revision together. An older provider can fail inventory verification on names it omits; it cannot publish an incomplete save. No merge, publication or deployment is included. ## Model Used OpenAI GPT-6 family through Codex, with reasoning, code editing, shell tools and automated tests. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 11:00:32 -06:00
Nicky LeachandClaude Opus 5.5	55ea9cfa1b	fix(runner): match retained cleanup and Codex close types to their contracts (#19495 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip Runner runs native agent sessions, such as Codex, behind typed contracts > - #18839 started to typecheck the Runner tests, and it listed contract types that do not match their runtime use > - First, retained runnerd cleanup accepts any backend kind, but cleanup works only when that kind is `"runner"` > - Second, `CodexHarnessSession.close` makes its input optional, but the `HarnessSession` contract requires a reason > - This pull request narrows both types to match their contracts > - The benefit is that the compiler now rejects a cleanup proof that would silently release nothing ## Linked Issues or Issue Description No issue exists. #18839 listed both items under its product questions and did not change them. What existing behavior does this improve? 1. `completeRetainedNativeSessionCleanup` builds its release key from `proof.backend.kind`. The quarantine key comes from the backend descriptor's `kind`, and every production backend uses `"runner"`. If the two kinds differ, the function returns 0 and the quarantine stays. `RetainedRunnerdMaintenanceInput` and `RetainedRunnerdCleanupProof` typed `backend.kind` as any string. 2. `HarnessSession.close` takes `{ reason: string; force?: boolean }`. `CodexHarnessSession.close` took `input?: { reason: string }`. Current behavior The only production caller passes `backend: { kind: "runner", name: "codex_app_server" }`. All production `close` calls pass a reason. So nothing fails today. Proposed behavior The types state the same rules as the contracts. Reason and benefit A future caller that passes another kind gets a compile error, not a quarantine that stays without notice. Breaking changes None for production callers. Both types are narrower, and every current caller already passes the narrow values. ## What Changed - `packages/paperclip-runner/src/live/runnerd-codex-transport.ts`: `backend.kind` is `"runner"` in `RetainedRunnerdMaintenanceInput` and `RetainedRunnerdCleanupProof`. `name` stays a string. - `packages/paperclip-runner/src/drivers/codex/codex-harness-session.ts`: `close` takes `{ reason: string; force?: boolean }`, as the contract does. The body still reads `input?.reason`, because a generated test script calls `close()` with no arguments. Codex closes the same way with or without `force`. - `packages/paperclip-runner/src/live/runnerd-contract-types.test.ts`: new type test. The Runner test typecheck fails if either type accepts another backend kind, or if the Codex `close` parameters differ from the contract. With the master types, this file has 4 type errors. Not in this pull request: #18839 also listed that `PrpEvent.normalizedSessionId` is required in the type but optional in the event schemas. A separate pull request handles it in `validatePrpEvent`. ## Verification - `pnpm exec vitest run src/live/runnerd-contract-types.test.ts` (in `packages/paperclip-runner`): passes 2 tests. - `tsc --noEmit` passes in `packages/paperclip-runner` for `tsconfig.json`, `tsconfig.surfaces.json`, `tsconfig.test.json` and `tsconfig.surfaces.test.json`. - `cd server && pnpm typecheck`: passes. The server builds the maintenance input with the `"runner"` literal. - In `packages/paperclip-runner`, after `pnpm run test:typescript:prep`: `pnpm exec vitest run src/live/runnerd-codex-transport-settlement.test.ts src/drivers/codex/codex-protocol-cleanup.test.ts src/drivers/codex/codex-app-server-driver.lifecycle.test.ts` passes 3 files and 43 tests. - Not run locally: the full Runner `pnpm test` (including Rust tests) and the root `pnpm build`. Required CI covers them. ## Risks - Low risk. The change affects types only, plus one comment in the Codex session. Runtime behavior does not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 16:54:09 +00:00
Nicky LeachandClaude Opus 5.5	21a1e78852	feat(environments): add managed plugin environment rows (#19490 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document > - That document cannot declare a plugin environment yet, so a managed instance cannot get a plugin-provided environment without manual setup > - The full change is split into three pull requests. This first one adds the plugin environment rows and their write floor, and nothing calls them yet > - The second pull request lets the managed config declare plugin entries and provisions them at boot. The third sets the instance default once and advertises the new managed-config capability > - The benefit is a smaller review for each step, and no instance changes behavior until the last step lands ## Linked Issues or Issue Description Subsystem affected Server environments (managed-config environment provisioning, environment routes) and the board UI's adapter Test target. Problem or motivation A managed instance cannot get a plugin environment from its managed config. Company settings cannot create one either. So each managed instance that needs one requires manual setup. Proposed solution Over three pull requests, let an `environments` entry in the managed config declare `driver: "plugin"`. Boot then provisions one locked row for each plugin entry, and makes one of them the instance default once on new instances. This pull request adds only the row layer: create or adopt a marked row, archive it, find it, and keep tenants from editing it. Alternatives considered Manual setup on each instance, and a new control-plane endpoint. Both need a step outside the deploy. The managed config already delivers the sandbox entry. Roadmap alignment This extends the existing managed-config `environments` section, which already provisions the managed sandbox. It adds no new product surface. ## What Changed - New `server/src/services/managed-plugin-environments.ts`, used through new `environmentService` methods (`ensureManagedPluginEnvironment`, `archiveManagedPluginEnvironment`, `findManagedPluginEnvironment`): - The ensure validates the driver config through `normalizeEnvironmentConfig` and `validatePluginEnvironmentDriverConfig`, the same path an operator-created plugin environment takes. - It marks the row `managedByPaperclip: true` and `managedPluginEnvironment: "<pluginKey>:<driverKey>"`. - It adopts an existing unmarked row for the same plugin and driver instead of adding a second one, so a hand-made row keeps its agents. - A row that already matches gets no write. Changes write activity log entries. - Archive sets an active marked row to `archived`. The next ensure reactivates it. - Migration `0437`: unique index `environments_managed_plugin_environment_idx` on `metadata ->> 'managedPluginEnvironment'`. This is the only schema change of the three pull requests. Snapshot `0432` is pruned to keep the five newest. - `server/src/routes/environments.ts`: `isPlatformProvisionedEnvironment` also covers the new marker (write floor and redaction). `managedPluginEnvironment` is a reserved marker key, so a client cannot stamp it on its own row. Nothing provisions marked rows yet, so the marker-clear escape hatch stays open for them. The next pull request refuses it while the managed config declares the row's entry. - `ui/src/lib/adapter-test-environment.ts`: `resolveManagedSandboxEnvironmentId` requires `driver === "sandbox"`. The plugin row also carries `managedByPaperclip`, so the old "not local" check could take it for the managed sandbox. ## Verification - `cd server && npx vitest run src/__tests__/managed-plugin-environments.test.ts` (embedded Postgres): - The ensure creates the marked row through the plugin validator. A second ensure writes nothing and logs nothing. - A hand-made row for the same plugin and driver is adopted. It keeps its id and its agents. - Archive, then ensure, reactivates the row. - The ensure refuses while the plugin has no running worker, and refuses a name that another environment holds. - The unique index allows one row per marker. - `npx vitest run src/__tests__/environment-routes.test.ts`: on a cloud-managed instance, edit, env-var edit, status change and delete of a marked plugin row are refused, including for instance admins. The marker alone keeps the floor. A client cannot stamp the marker. The marker clear is allowed. - `npx vitest run src/__tests__/environment-service.test.ts`: passes. - `cd ui && npx vitest run src/lib/adapter-test-environment.test.ts`: the plugin row is never taken for the managed sandbox. Under managed-sandbox-only, an agent on the sandbox tests in the sandbox, and an agent on the plugin environment tests there. - `pnpm --filter @paperclipai/server typecheck`, `pnpm --filter @paperclipai/ui typecheck` and `pnpm --filter @paperclipai/db check:migrations` pass. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. `pnpm verify` asks for broader verification because of the migration. ## Risks - Migration: one new partial unique index on a small table. No row carries the marker before this change, so the index cannot fail on existing data. - No caller uses the new ensure yet, so no instance changes behavior. - The UI change affects only lists that hold a non-sandbox row with `managedByPaperclip`. Today that is only the local row, which the old check already excluded. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 16:52:43 +00:00
DottaandClaude Opus 5.5	871a9b3180	test(restart-chaos): add Phase 1 control-plane restart chaos harness (#19477 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane owns task execution, recovery and Retry for every agent run > - Server restarts were the largest remaining source of stalled tasks in the staging soak campaign > - The soak could only observe restarts that staging happened to perform, so a failure could not be repeated > - This pull request adds a script harness that causes the restarts on purpose and grades each task on fixed rules > - The benefit is a repeatable restart failure with a replay command and an evidence bundle for each fixer ## Linked Issues or Issue Description What existing behavior does this improve? The restart test coverage. Today the runner-e2e restart cells test one fixed restart point per cell. They cannot run many tasks at once, cannot choose a random fault time, and cannot replay a failure. Subsystem affected Test tooling (`tests/restart-chaos/`). It reuses the runner-e2e process code in `tests/runner-e2e/`. Product code does not change. Current behavior Restart robustness is checked through a small set of fixed cells. A random restart during a real task has no defined pass rule. Proposed behavior A harness starts an isolated control plane and seeds a "Chaos Lab" company. It launches seeded workloads, injects K1, K2 or K4 faults at seeded times, and grades each task on rules R1 to R6 plus a deterministic task oracle. `--replay <wave>/<task>` repeats one task. Reason and benefit Fixers get a replay command and an evidence bundle for each failure signature. The first smoke wave already found three product signatures (dead Retry after a restart, a false over-budget stop after restarts, and an OpenCode turn killed mid-turn that ends as a provider failure). Breaking changes None. The change adds new test files, one entry in the test-project typecheck list and a gitignore entry. ## What Changed - Add `tests/restart-chaos/`: - `supervisor.ts` owns one isolated server. It reuses `replaceController`, the process-tree stopper, the environment isolation helpers, the config seed, the port reservation, the wrapper-owned Postgres and `server-entry.ts` from `tests/runner-e2e/`. It does not copy them. Faults: K1 (SIGKILL, immediate restart), K2 (SIGTERM, restart) and K4 (SIGKILL, 2 to 5 minute hold). Postgres and the controller runtime cache stay across a fault. - `seed.ts` creates the company, encrypted provider secrets, native Codex, ACPX Claude and OpenCode agents from the runner-e2e catalog, and the `chaos-sandbox` project. The project is a git repository with a Python CLI, one failing test and a four-minute `bench.sh`. - `scheduler.ts` makes a seeded plan of workload, adapter, fault, fire time, hold and nonce for each task. It saves the plan so that `--replay` can run one task again. - `observer.ts` polls each issue and its runs. When a task stops and offers Retry, it sends `POST /api/issues/:id/retry` one time, with the same body as `useTaskRetry.ts`. - `grader.ts` grades R1 to R6 as pure functions. `workloads.ts` defines workloads A1, B1, B5 and C1 and their oracles. - `evidence.ts` and `ledger.ts` write an evidence bundle for each failure, a JSONL ledger and `SUMMARY.md`. All results go to the gitignored `tests/restart-chaos/results/`. Known provider key values are redacted. - Unit tests: `scheduler.test.ts` (determinism, replay and CLI parsing) and `grader.test.ts` (R1 to R6 and the oracles on fixture timelines). These tests need no credentials. - Add `tests/restart-chaos` to the test-project typecheck list (`scripts/typecheck-test-projects.mjs`), with a `tsconfig.layers.json` like `tests/runner-e2e`. The harness runs through direct script invocations, not `package.json` scripts. - After the first Greptile review: - A failed fault or an unexpected server exit now gets one bounded recovery. If the recovery fails, the wave fails and does not hang. - Incomplete cleanup is now a harness error, and the harness keeps the instance. - After the C1 follow-up comment, the observer needs a fresh rest window before it grades the task. - `--replay` now fires the recorded faults that crossed the task, at the same offsets, including the faults that other tasks requested. - A failed workspace-operation read is now a sweep error. It cannot give a clean rest. - R2 now checks that workspace files from before the first fault still exist at rest. - B5 now needs the completion record that `bench.sh` writes for each task. - After the second round of reviews: - An idle `todo` or `in_progress` task with no live run and no named wait is no longer treated as resting. The observer keeps watching it until the task resumes or passes the R5 stall limit. - A failed injected fault now fails the wave with exit 2. - T-oracle commands run agent-editable code. They now run under a macOS Seatbelt sandbox with a scratch `HOME` and no network, and they cannot read the operator's home directory. On other platforms these commands are refused unless the operator explicitly allows them. - R4 now counts every active run. A wake comment may be served again only after a run that did not succeed. - Evidence redaction now covers every environment variable name that ends in `KEY`, `TOKEN`, `SECRET` or `PASSWORD`. - If the pre-fault snapshot fails, only R2 for that task is left unchecked. The wave continues. - B5 also needs bench progress that the harness observed while it polled. - Add `doc/plans/2026-11-06-control-plane-restart-chaos.md` and its campaign ledger. ## Verification - `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`: 38 tests pass. The tests include observer tests with a fake API, a real `bench.sh` run, and a sandbox test that cannot read a canary secret. - `node scripts/typecheck-test-projects.mjs tests/restart-chaos`: passes. - `server/scripts/typecheck-layers.test.mjs`: 45 tests pass, including "covers every project under tests/". - A supervisor check with no credentials passed. K1 was ready again in about 6 seconds. K2 with a 3-second hold was ready again in about 9 seconds. Company data stayed readable. - One paid smoke wave ran: `--env local --adapters codex,claude,opencode --workloads A1,B1,C1 --faults K1 --tasks 3 --concurrency 2 --seed 7`. The harness worked from start to end, and all three tasks crossed a restart. The 0/3 result comes from product failures, which the campaign ledger records. A scan found no provider key in the results. - To run a wave: ```sh pnpm --filter @paperclipai/paperclip-runner build:runner-binaries set -a; source ~/.secrets; set +a node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --env local --adapters codex,claude,opencode --concurrency 4 --tasks 8 --faults K1,K2,K4 --seed 42 node cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --replay <wave>/<task> ``` ## Risks - Low product risk. The change adds test tooling only and does not change product code. - A wave spends provider money. The company budget hard cap (`--budget-cents`, default 3000) is the only spend limit. The plan's per-wave and per-night refusal is not implemented yet. - Known gaps: - `--env daytona` is reserved. It fails with a TODO in `seed.ts`. The Daytona sandbox sweep for R6 is not implemented yet. - The project requests isolated git-worktree workspaces. A B1 task still ran in the shared fixture repository. Concurrent B1 and B5 tasks can share files. The oracles use marker files for each task. - Only K1, K2 and K4 at seeded random times are available. Boundary triggers (B1 to B16), other faults and the UI probe come in Phase 2. - The paid smoke did not exercise B5 or K4. - On macOS, B1 oracle commands need `python3` outside the operator's home directory, for example `/usr/bin` or Homebrew. On Linux they are refused until a sandbox is added. - The B5 proof is best-effort against a deliberately adversarial agent, because that agent can write to the workspace. - Startup phase timings come from polling `/api/ready`. The server records its own timings only as OpenTelemetry spans. - `pnpm verify` and the CI Vitest jobs do not select these unit tests. The root Vitest projects cover workspace packages only. CI typechecks the files, but the tests run only through the vitest command above. - After the smoke, rest detection and the review fixes changed. A recovery action that the board owns now counts as a rest, and R6 reports it. A paid wave did not run again after these changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used in Claude Code as an agent with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:45:52 -06:00
DottaandClaude Opus 5.5	819420a60a	fix(runner): give a cold OpenCode restore the 60 s admission budget (#19487 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native OpenCode agents run through a local runner. After a control-plane restart, a replacement runner restores the saved provider session. > - The replacement runner does run preparation, `opencode serve` startup and session resume inside one `run.attach` command. > - The controller waits only 30 s for that command. A fresh start gets separate bounds for `run.prepare` and `session.open` for the same work. > - On a loaded host, a cold OpenCode restore took 10-38 s. When it took more than 30 s, the continuation failed with `runner_local_connect_failed: PRP command run.attach timed out`. > - This pull request gives a cold OpenCode restore the same 60 s cold-admission budget that Pi already uses. > - The benefit is that a task that waited on a question across a restart continues when the user answers, without a failed run. ## Linked Issues or Issue Description What happened? A native OpenCode agent asked a structured question. The control plane restarted while the task waited. The user answered. The continuation run failed with `native_session_recovery_failed: Error: runner_local_connect_failed: PRP command run.attach timed out`, about 35 s after the runner transport connected. This happened in 1 of 2 runs of the `local-session-integrity.runner-opencode.local.structured-question-restart-resume` runner E2E cell. Expected behavior The continuation restores the saved OpenCode session and finishes the task. A restart must not cause a failed run. Steps to reproduce 1. Run `pnpm test:e2e:runner -- --max-automatic-retries 0 --id local-session-integrity.runner-opencode.local.structured-question-restart-resume` on a host with other load. 2. The case asks a question, restarts Paperclip and answers the question. 3. When the cold restore takes more than 30 s, the continuation run fails at `run.attach`. Paperclip version or commit `master` at `0252421d5c`. Evidence from the runner spans (`runner.session.startup` of the continuation run): 31.0 s (passed, close to the bound), 38.0 s (failed), 14.0 s and 10.1 s (passed). The first (fresh) runs took 4.9-13.7 s. ## What Changed - `coldAdmissionTimeoutMs` returns 60 s for a cold OpenCode runner process. Live adoption (a warm runner) keeps 30 s. Other providers keep 30 s, except Pi, which already had 60 s. - The unit test for the cold admission budget covers the OpenCode cold and live cases. ## Verification - Red before the fix: `cd packages/paperclip-runner && npx vitest run src/live/runnerd-codex-transport.test.ts -t "cold process admission budget"` failed on `coldAdmissionTimeoutMs("opencode", undefined, true)` (30000, expected 60000). - Green after the fix: the same command passes. - Live runner E2E on macOS arm64 with OpenRouter (`openrouter/deepseek/deepseek-v4-flash-0731`), local runner, at the fixed revision. Command: `pnpm test:e2e:runner -- --max-automatic-retries 0 --id local-session-integrity.runner-opencode.local.structured-question-restart-resume`. - Before the fix: 1 of 2 campaign runs failed at `run.attach` (restore 38.0 s). The passing run restored in 31.0 s. Two more local runs passed (restore 14.0 s and 10.1 s). - After the fix: 4 of 4 runs that reached the restart passed (restore 6.2 s, 11.3 s, 5.1 s and 11.8 s). One more run failed before the restart because the model sent malformed `request_human_input` arguments. That failure is not related to this change. - Limitation: host load was lower during the after-fix runs, so no restore took more than 30 s. The unit test proves the new bound. - Other failures in `runnerd-codex-transport.test.ts` are unchanged on `master` in this local environment (the fake Rust fixture binary is not built locally). CI builds it. ## Risks - Low risk. The change only makes a cold OpenCode admission wait longer before it reports a failure. A runner that is really stuck now fails after 60 s instead of 30 s. - A fresh OpenCode start now shares one absolute 60 s deadline for `session.open` and provider identity, the same as Pi. Fresh OpenCode starts took 5-14 s. - Inner bounds stay in place: the runner still bounds each provider request (30 s) and OpenCode health checks (10 s). ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits). Reasoning mode was on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:33:51 -06:00
Nicky LeachandClaude Opus 5.5	fcf5dc2395	fix(ai-connections): reject unknown binding modes before grant checks (#19494 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run on AI accounts that people connect as AI connections > - Before a run, `aiConnectionService.select` resolves the agent's binding to one account grant and checks who may use it > - The grant-kind checks cover only the `responsible_user`, `shared` and `delegated` modes > - Every caller parses the binding first, so no other mode reaches `select` today. But if one did, it would skip the grant-kind checks > - This pull request rejects any other mode before `select` reads any data > - The benefit is that an authorization check cannot be skipped through a bad binding in the future ## Linked Issues or Issue Description No issue exists. The new test typecheck in #18739 showed that a test passed `mode: "connection"`, which is not a valid mode. #18739 fixed the test and listed the production question. What existing behavior does this improve? `select` in `server/src/services/ai-connections.ts` reads `binding.grantId` for every mode except `responsible_user`. It checks the grant kind only when the mode is `shared` or `delegated`. Current behavior All runtime callers parse the binding with `aiRuntimeConnectionBindingSchema` or `aiConnectionBindingSchema` first (`heartbeat.ts`, `agent-harness-verification.ts`, `routes/agents.ts`). So no live path sends an unknown mode. If a future caller skipped parsing, an unknown mode would pass with no grant-kind check. Proposed behavior `select` rejects an unknown mode with the existing `ai_connection_incompatible` failure (422), before it reads any data. Reason and benefit A guard in the owner of the check is cheaper than trusting every future caller. Breaking changes None. Valid bindings behave the same. ## What Changed - `server/src/services/ai-connections.ts`: `select` throws `ai_connection_incompatible` for any mode other than `responsible_user`, `shared` or `delegated`. - `server/src/__tests__/ai-connection-binding-mode.test.ts`: new test. It sends `mode: "connection"` and expects a 422 with no database read. ## Verification - `pnpm exec vitest run server/src/__tests__/ai-connection-binding-mode.test.ts`: passes. Without the service change, the same test fails because `select` reads the database. - `pnpm exec vitest run server/src/__tests__/ai-connections.test.ts server/src/__tests__/agent-hire-ai-connections.test.ts server/src/__tests__/ai-connection-binding-mode.test.ts`: 3 files, 243 tests pass. - `cd server && pnpm typecheck`: passes. This includes the source layers and the three test-tree shards. - Not run locally: the full `pnpm test:run` and the root `pnpm build`. Required CI covers them. ## Risks - Low risk. The new check only rejects bindings that the schemas already reject. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 16:28:25 +00:00
Nicky LeachandClaude Opus 5.5	e1332b997f	fix(plugins): align tool run context and interaction input types (#19493 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins add tools and issue-thread interactions through `@paperclipai/plugin-sdk` > - The SDK types are the public contract between the host and plugin authors > - Two parts of that contract do not match what the host does at runtime > - First, `ToolRunContext.projectId` says it is always a project UUID. The tool gateway sends an empty string when a run has no project. The direct tool route rejects such runs with 400 > - Second, three interaction helpers ask for the parsed output type, so plugin authors must pass fields that the host fills in with defaults > - This pull request makes the types and both tool paths agree with the runtime > - The benefit is that agents with no project can run plugin tools, and plugin authors can omit the fields that have defaults ## Linked Issues or Issue Description Fixes #3273 Related: #3274 removes the `projectId` requirement from the route. That change alone is not enough today: the route now also looks up the project to check its company, and that lookup needs a value. This pull request also skips that lookup when the run has no project. It gives plugins one value for "no project" on both tool paths. The new test typecheck in #18739 and #18839 also listed both problems as possible product problems. ## What Changed - `server/src/routes/plugins.ts`: `POST /api/plugins/tools/execute` accepts a `runContext.projectId` that is missing, `null`, or an empty string. It passes an empty string to the plugin, the same value that the tool gateway sends. It checks the project's company only when a project ID is present. A `projectId` that is not a string returns 400. - `server/src/routes/openapi.ts`: the request schema marks `runContext.projectId` as optional and nullable. - `packages/plugins/sdk/src/types.ts`: - The `ToolRunContext.projectId` comment says the value is an empty string when the run has no project. - `suggestTasks`, `requestConfirmation` and `requestCheckboxConfirmation` take `CreateIssueThreadInteractionInput`, as `askUserQuestions` already does. The host parses this input with the schema, so the defaults apply. - `packages/plugins/examples/plugin-kitchen-sink-example/src/worker.ts`: the example passes `undefined` instead of an empty project ID to `issues.create`. - `doc/plugins/PLUGIN_SPEC.md` (section 13.10): documents the empty project ID. - Tests: - `server/src/__tests__/plugin-routes-authz.test.ts`: tests a run with an empty, missing, or `null` project ID, which skips the project lookup. Also tests that a project ID that is not a string returns 400. - `packages/plugins/sdk/tests/testing-actions.test.ts`: removes the cast that the output type required. ## Verification - `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/openapi-routes.test.ts packages/plugins/sdk/tests/testing-actions.test.ts`: 3 files, 81 tests pass. - `cd server && pnpm typecheck`: passes. This includes the source layers and the three test-tree shards. - `tsc --noEmit` passes for `packages/plugins/sdk` (`tsconfig.json` and `tsconfig.test.json`) and for the kitchen-sink example. - Not run locally: the full `pnpm test:run` and the root `pnpm build`. Required CI covers them. ## Risks - Low risk. The direct tool route now accepts calls that it rejected before. It still checks that the agent and the run belong to the company. With no project ID, there is no project to check. - A plugin that reads `runCtx.projectId` can get an empty string on both tool paths. The tool gateway already sent this value before this change. Only the documentation is new. - The SDK parameter types are wider. Existing plugin code still compiles. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits) and extended thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 16:25:43 +00:00
Devin FoleyandPaperclip	7840d894fa	fix: retain files after accepted remote copy-back exhausts retries (#19491 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote workspaces keep unsaved files bound to the run that wrote them. > - An accepted result can exhaust its copy-back retries before those files are saved. > - Failed-workspace preservation rejects every accepted result, so a stopped writer can keep the workspace held after a newer setup attempt fails. > - This pull request lets the existing owner retain files from that exact stopped remote writer. > - File recovery can then release the hold without applying the old result or repeating its provider work. ## Linked Issues or Issue Description What happened? A failed native remote writer had an accepted result and exhausted copy-back retries. Its sandbox was stopped with a valid retained-source receipt. A newer attempt failed on the writer's unsaved workspace. Task Retry could not register the missing file-recovery obligation because preservation returned as soon as it found the old accepted result. Expected behavior Task Retry must let the existing recovery owner save the exact retained files. The accepted result, failed attempts and current task must stay intact. The old provider work must not run again. Steps to reproduce 1. Finish native work in a remote task workspace and accept its completion result. 2. Exhaust copy-back retries, stop and retain that exact sandbox, and leave the workspace dirty. 3. Let a newer setup attempt fail on the old writer's hold. 4. Use ordinary task Retry. Before this fix, the missing file obligation is never recorded. Paperclip version or commit Reproduced against master at `51a65af9b81f3c60f8210047e9a3eeaada7bc155` with isolated PostgreSQL and real files. Related: #19385 preserves accepted copy-back interrupted by Stop. #19422 admits saved input after verified recovery. #19464 and #19466 repair preparation and Retry continuation. This change covers an older remote writer whose accepted copy-back has exhausted its retries. ## What Changed - Admit only terminal, exhausted accepted copy-back on a failed remote writer with a finished run and no live controller or finalizer. - Require the exact stop-and-retain receipt, accepted completion contract and current dirty workspace binding. Keep native-sync sources on their existing export path. - Use the existing file-preservation recorder and worker. Add no new recovery worker, schema, polling or replay path. - Cover bound and orphaned entry points, reusable leases, changed proof and the ordinary HTTP Retry journey. Document the recovery behavior. ## Verification - Causal baseline: the corrected bound-writer and orphaned-lease tests both fail without the source change and pass with it. - Assembled acceptance: authenticated HTTP Retry records the old writer's missing obligation; the real recovery service saves its files; concurrent continuation calls queue exactly one successor for the newer failed setup. PostgreSQL, filesystem, tar, Git and the API/service path are real. Provider transport is controlled, and no model turn is dispatched. - `pnpm check:module-boundaries`, `pnpm -r typecheck`, `pnpm build` and `git diff --check` pass. - Four full affected recovery suites pass: 659 tests. Command: `pnpm --dir server exec vitest run --config vitest.config.ts src/services/workspace-recovery.test.ts src/services/native-runtime/native-failed-workspace-recovery.test.ts src/services/native-runtime/native-cancelled-workspace-preservation.test.ts src/services/native-runtime/native-workspace-export-retry.test.ts`. This includes the final 16 new regression, refusal and assembled journey cases. - `pnpm verify --explain` completed. It selected broad transitive coverage; this is planning evidence only. Hosted CI will supply the remaining broad coverage. - Dependency cost: extra indexed source/result reads run only for this exact terminal retained-result path, inside the existing bounded transaction. Existing notifications activate the existing recovery owner. No idle scan is added. - Fixture failures are accounted for: the first fixture used an active-only stop helper on a released lease. The corrected fixture produced the causal baseline above. One negative fixture tried to remove a contract while an accepted result referenced it and failed the database foreign key. That impossible fixture was removed; the positive journey passed in the same run. Neither fixture failure is reported as a product regression or a passing full run. - Hosted CI is fully green at the reviewed commit: 54 successful checks and 3 intentional skips, with no incomplete or failed checks. Greptile is 5/5 with no unresolved threads. - Hosted rollout and a live task continuation remain pending. Local API admission does not prove a deployed model run. ## Risks This expands file preservation for a narrow terminal failure. Wrong source identity could retain unrelated files; unchanged writer, epoch, location, provider lease, stop receipt and accepted contract checks fail closed. Active ownership, unsafe archives, rejected results and native-sync sources stay on their existing paths. Existing recovery owns physical source locking, copy verification and bounded retries. It preserves unknown action outcomes and leaves the accepted result uncommitted. There is no migration or configuration change. ## Model Used OpenAI Codex, `gpt-6.1-sol`, xhigh reasoning effort. Context window size is unavailable. Used repository tools, isolated tests and code execution. No delegated agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 08:23:36 -08:00
Nicky LeachandPaperclip	5e62d8ee82	Report the native private host-control protocol version (#19488 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner executes agent work and reports its binary contract through build metadata. > - Unix Runner already supports a private host-control socket for park, claim, and retire requests. > - A host cannot identify that protocol from the current metadata. > - This PR adds an explicit version field for that existing private protocol. > - Hosts can verify the selected binary before opening the private channel. ## Linked Issues or Issue Description Refs #19140 and #19129, which added the existing native lifecycle and private channel. What existing behavior does this improve? The `paperclip-runnerd --build-metadata` response. Current behavior The native binary implements the private fd 4 protocol on Unix. Its metadata has no field that identifies this protocol. Recovery versions describe different contracts. Proposed behavior Report `privateHostControlVersion: 1` on Unix builds. Omit the field on other builds. Keep ordinary artifact checks and runtime behavior unchanged. Reason and benefit A host can verify protocol support on the selected binary before using the channel. This field does not mean a provider is ready to park or that a host implements public release/resume or qualified sleep/wake. Breaking changes The JSON response gains one optional field on Unix. Current artifact parsers and launch checks accept extra fields. Binary contract version 2 stays unchanged. ## What Changed - Add the Unix-only private host-control metadata version. - Check the exact version and non-Unix absence in metadata unit and real CLI tests. Keep hosted parking and public release/resume fields absent. - Add a regression test that proves ordinary artifact parsing accepts and ignores the new field. - Document the field, exact-version checks, and separate provider and host qualification. ## Verification - On macOS arm64 with Rust 1.97.1, passed narrow offline, locked Cargo tests for the metadata unit, six existing private-protocol cases, wrong-fd CLI rejection, and real CLI metadata output. - Passed `cargo fmt --manifest-path packages/paperclip-runner/runner/Cargo.toml --all -- --check`. - Passed `vitest run src/evals/runnerd-artifact.test.ts` from the Runner package: 5 tests. Initial collection could not resolve Ajv in the fresh worktree. Connecting existing local dependencies resolved that setup failure; no source change was needed. - `git diff --check` passed. Reviewed the diff for secrets, personal data, and private references. - GitHub CI passed on commit `0790af690afaff10f3b69fd9130b5be1c5bab38c`: [main PR workflow](https://github.com/paperclipai/paperclip/actions/runs/42248006277), [Docker Runner check](https://github.com/paperclipai/paperclip/actions/runs/42248006207), and [Windows Runner](https://github.com/paperclipai/paperclip/actions/runs/42248006518). Windows tests cover omission of the field on non-Unix builds. All 61 current-head check runs settled without failure; the latest status context passed. - [Standard Greptile review](https://github.com/paperclipai/paperclip/pull/19488#issuecomment-6689260747) scored that same commit 5/5 with no actionable findings or unresolved threads. - No live provider or host sleep/wake test was run for this metadata-only change. ## Risks The main risk is treating protocol support as permission to park a provider or as proof that a host supports release/resume. The documentation keeps those checks separate. No caller enables the channel from this field in this PR. Current parsers check named fields or project known fields, so the optional field does not change ordinary artifact checks. No migrations, new dependencies, runtime flags, checkpoints, or provider pins. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), High reasoning, through Codex with tool use and code execution. Independent agent review checked the metadata boundary and compatibility. The context window size is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 08:18:03 -08:00
DottaandClaude Opus 5.5	e2f0fc9e7d	fix(runner): end the detached Codex controller's event stream so graceful shutdown exits promptly (#19483 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane runs native Codex turns through the Paperclip runner, and a restart must hand those turns to the next controller > - On a graceful shutdown (SIGTERM), the server detaches each native Codex session and leaves the runner alive for restart recovery > - After the detach, the shutdown waited the full 30 s "bounded heartbeat execution finalizer drain" every time > - A supervisor with a 30 s graceful window (for example the runner-e2e supervisor) then sent SIGKILL, so a graceful restart became a hard kill > - This pull request ends the detached controller's event stream, so the old run execution unwinds immediately > - The benefit is that graceful restarts and deploys exit in about one second and stay graceful ## Linked Issues or Issue Description What happened? Every graceful shutdown with an active or restart-suspended native Codex run waited 30 s before exit. The server log showed `graceful heartbeat run drain complete` with the run in `restartSuspendedRunIds`, then `bounded heartbeat execution finalizer drain timed out` 30 s later. Under a supervisor with a 30 s grace period, the server got SIGKILL. Steps to reproduce 1. Start a native Codex run (runner-e2e cell `everyday-workflows.runner-codex.local.recover-controller`). 2. Stop the server with SIGTERM while the turn is active (`PAPERCLIP_RUNNER_E2E_CONTROLLER_STOP_SIGNAL=SIGTERM`). 3. See that the server does not exit until the 30 s drain timeout, or until SIGKILL. Expected behavior The server exits promptly. The detached run is left for restart recovery. Root cause - `drainRunningRunsForShutdown` (`server/src/services/heartbeat/recovery.ts`) calls `detachNativeSessionsForRestart` for the native run. This calls `CodexHarnessSession.detachControllerForRestart`. - `CodexHarnessSession.detachControllerForRestart` (`packages/paperclip-runner/src/drivers/codex/codex-harness-session.ts`) detached the transport but did not close the session `eventQueue`. - The transport closed its notification queue. `pumpNotifications` then returned normally and did not close the session queue. - `executeNativeSession` waited on `eventIterator.next()` for an event that could never arrive. The heartbeat execution promise (registered at `server/src/services/heartbeat/queue.ts` in `activeRunExecutionPromises`) never settled. - `drainActiveRunExecutions` awaited that promise until the 30 s bound in `drainRunExecutionFinalizersForShutdown`. Temporary diagnostics confirmed this: during the hang, the only pending entry was the `queue.ts` run execution, in the `execution.runtime` phase, with no pending wakes. Other drivers (Cursor Cloud, external runnerd) already end their event stream on detach. ## What Changed - `CodexHarnessSession.detachControllerForRestart` now clears and closes the session event queue after the transport detaches. The detached consumer gets end-of-stream. The executor then throws `NativeControllerDetachedForRestartError`, and heartbeat leaves the run for restart recovery. - Buffered events are not delivered to the detached consumer. The next controller replays them from the runner journal. - Pending runtime requests stay open for the adopting controller (no `cancelPendingRequests`). - If the transport detach fails, the queue stays open. The existing uncertain-ownership path is unchanged. - Added a regression test: after detach, the session event iterator ends, and a late provider notification does not reach the detached consumer. ## Verification - Red, then green: `npx vitest run src/drivers/codex/codex-app-server-driver.lifecycle.test.ts` in `packages/paperclip-runner`. Before the fix, the new test failed with `hung`. After the fix, all 22 tests pass. - `npx vitest run src/drivers/codex src/backends`: 27 files, 457 tests pass. - `tsc --noEmit` for `@paperclipai/paperclip-runner` passes. - Live: `PAPERCLIP_RUNNER_E2E_CONTROLLER_STOP_SIGNAL=SIGTERM pnpm test:e2e:runner -- --max-automatic-retries 0 --id everyday-workflows.runner-codex.local.recover-controller` - Before (unchanged code, reproduced locally): `Paperclip did not stop within 30000ms; sending SIGKILL`, `server exited code=null signal=SIGKILL`. - After, run 1: PASS. Drain complete with the run in `restartSuspendedRunIds`, then `server exited code=0 signal=null` within the same second. The next server resumed the run and the task completed. - After, run 2: PASS. Same clean exit, no SIGKILL. - `pnpm verify` returned exit 2 (broader verification required). Required CI supplies that coverage. - Note: 21 tests in `src/live/runnerd-restart-continuation.test.ts`, `runnerd-controller-outage.test.ts` and `runnerd-codex-transport-warm-attachment.test.ts` fail on this machine with and without this change. They use the transport directly, not `CodexHarnessSession`. ## Risks - Low risk. The change only affects a session after its controller detaches for restart. That controller has no authority to settle the run. - The 30 s finalizer bound stays in place. It still applies when a run execution is past the provider stream, for example while it persists an accepted result or settles a workspace. Those durable writes must finish (or reach their recovery records) before exit, so waiting for them is correct. - Dropped buffered events: the old consumer could not persist them anyway after the transport closed. The next controller replays from the runner journal. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edit, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 10:08:20 -06:00
DottaandClaude Opus 5.5	53645a0104	perf(workspaces): record per-phase timings for workspace saves (#19472 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native runs in a sandbox save the workspace to object storage. A run saves before it starts work and again at the end. > - A save of a large repository takes 150 to 250 seconds when it runs alone. When several large saves overlap, each save takes 510 to 900 seconds. > - A slow save before the run starts delays the agent. It also gives a deploy a wider window to interrupt the run. > - The save operation records only a start time and an end time. It does not show which phase uses the time: in-sandbox capture, provider download, re-hash, encrypted upload, or publication. > - Without that data, a speed fix is a guess. Each possible bottleneck needs a different fix. > - This pull request records a bounded duration for each phase in the save operation's local metadata. > - The benefit is that one busy window on a deployment identifies the bottleneck. Then a targeted fix can be measured. ## Linked Issues or Issue Description Is your feature request related to a problem? Saves of large repository workspaces are slow, and a save does not record where the time goes. On one deployment, saves of a 460MB workspace archive (a full repository with history) took 150 to 250 seconds when they ran alone. During a window where eight large saves overlapped, each save took 510 to 895 seconds. Saves of small repositories in the same window stayed at 2 to 6 seconds. The operation row has only `startedAt` and `finishedAt`, so the contended resource is unknown. It can be the sandbox CPU, the provider download, or server disk and network. Describe the solution you'd like Each save invocation records per-phase durations and the archive size in the operation's metadata, for both outcomes. Describe alternatives you've considered Changes to the speed of a single phase without phase data. A local profile of one capture showed that `git fsck --full` on the archived history uses about half of the capture time. But full object integrity is a tested requirement, and the cheaper `--connectivity-only` check does not meet it. A larger delta-base cache gave no gain. A threaded check helps only on sandboxes with more than one core. Also, the overlap slowdown can come from the server, not the sandbox. A fix chosen without phase data could miss the real bottleneck. Related: #19444 (skip redundant saves on follow-up runs), #19458 (overlap file hashing in capture), #19462 (bounded bootstrap retry after a closed runtime listener). ## What Changed - `captureRemoteWorkspaceArchive` accepts an optional `timing` record. It writes `captureMs` (helper install and in-sandbox capture), `transferMs` (provider download), `verifyMs` (server re-hash), `byteSize`, and `entries` as each phase ends. - `saveRemoteWorkspace` adds `uploadMs` (encrypt, hash, and store) and `publishMs` (manifest read and authority commit). It also sets `reusedUpload` when it reuses an earlier upload. - `settleSaveInvocation` writes a `saveTiming` record (`paperclip.remote-workspace-save-timing.v1`) with `outcome`, `totalMs`, and the finished phases. It writes the record on success and on failure. A failed save keeps the phases that it finished. - The record holds only numbers, a fixed outcome, and a timestamp. It holds no paths or provider text. It stays in the instance database (`workspace_operations.metadata`), so it is not Telemetry. ## Verification - Extended `retains the writer and previous snapshot when object storage fails` in `server/src/services/remote-workspaces.integration.test.ts`. After the storage failure, the `final` operation has a `failed` timing with capture, transfer, and verify phases and no upload phase. After the retry, it has a `succeeded` timing with every phase. - Red/green: with the source change reverted, the test fails (`expected [ undefined ] to deeply equal [ { …(9) } ]`). With the change, it passes. - `npx vitest run src/services/remote-workspaces.integration.test.ts -t "object storage fails"` passes. - `npx tsc --noEmit -p server` passes. - A reviewer can read the record through `GET /api/heartbeat-runs/:runId/workspace-operations` (`metadata.saveTiming`). ## Risks - Low risk. The change adds metadata only. It does not change save order, fencing, retries, or archive bytes. - The latest settled invocation replaces `saveTiming`. Concurrent invocations of the same boundary keep only the last record. That is acceptable for diagnosis. - `performance.now()` calls add negligible cost. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (shell, browser for read-only staging GETs, code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> canary/v2026.1106.0-canary.16	2026-11-06 15:54:13 +00:00
DottaandClaude Opus 5.5	a434edf991	fix(runner-e2e): treat macOS EPERM from a zombie-only process group as exited (#19470 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The runner E2E suite restarts the Paperclip server to prove that agent work recovers after a controller restart > - The restart supervisor in `tests/runner-e2e` owns the server process tree and escalates to SIGKILL when graceful stop takes too long > - On macOS, `kill(-pgid)` returns EPERM, not ESRCH, when every member of the group has exited but is not reaped yet > - The supervisor treated that EPERM as fatal, so a server that exited at the same moment as the SIGKILL escalation aborted the restart with "Owned tree cleanup incomplete: Error: kill EPERM" > - This pull request makes the supervisor re-inspect the process table on EPERM and skip the group only when no running member remains > - The benefit is that restart recovery cases no longer fail on a harmless exit race on macOS, while a real permission failure still fails cleanup ## Linked Issues or Issue Description No public issue exists. This describes the bug. What happened? The runner E2E cell `everyday-workflows.runner-codex.local.recover-controller` failed on macOS with failure class `cleanup_failure`: "Stopped waiting for isolated server restart ...: Owned tree cleanup incomplete: Error: kill EPERM". The server log shows that the graceful stop took 30 seconds, the supervisor sent SIGKILL, and the server exited with code 0 in the same instant. The restart did not start, so the case never checked recovery. Steps to reproduce 1. On macOS, fork a child that calls `setpgid(0, 0)` and exits. Do not reap it. 2. Call `killpg(child, SIGKILL)`. The kernel returns EPERM (errno 1), not ESRCH. After the reap, it returns ESRCH. 3. In the runner E2E supervisor, any group that becomes zombie-only between the `ps` inspection and the signal causes the same EPERM, which the code rethrows. Expected behavior A group whose members have all exited counts as stopped, the same as ESRCH. EPERM stays fatal while a running member remains. Environment macOS 26.5 (Darwin 25.5.0), arm64, Node 26.4. The behavior is the same with and without the Claude Code sandbox. ## What Changed - `tests/runner-e2e/process-tree-owner.ts`: on EPERM from a group signal, read a fresh process table. Skip the group (as with ESRCH) only when it has no running (non-zombie) member. Otherwise rethrow. `signalSnapshot` is now async. - `tests/runner-e2e/process-tree-owner.test.ts`: two regressions. One covers a group that becomes zombie-only when signaled. One proves that EPERM stays fatal while a running member remains. - `tests/runner-e2e/README.md`: document the EPERM rule next to the existing zombie rule. ## Verification - Red before the fix: `node node_modules/vitest/vitest.mjs run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/process-tree-owner.test.ts` failed with `promise rejected "Error: kill EPERM"`. - Green after the fix: `process-tree-owner.test.ts`, `server-stop.test.ts`, and `process-tree.test.ts` pass (35 tests). `pnpm test:e2e:runner:typecheck` passes. - Kernel behavior: a Python fork probe on this Mac shows that `killpg` on an unreaped zombie group returns EPERM, both inside and outside the sandbox. - Live cell: `pnpm test:e2e:runner -- --max-automatic-retries 0 --id everyday-workflows.runner-codex.local.recover-controller` on macOS. The fixed revision passed. The server stop escalated to SIGKILL, the replacement server started, and the run recovered. See the comment below for the second rerun. ## Risks - Low risk. The change is in the test harness only. - EPERM is ignored only after a fresh inspection shows that no running process holds that group ID. A process group ID cannot be reused while zombie members still hold it, so the supervisor does not signal or skip an unrelated group. - The extra `ps` read runs only on EPERM. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). Default context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 09:48:52 -06:00
DottaandPaperclip	eb5c24b61c	perf(chat): speed up cached returns and preserve bottom follow (#19452 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat shows saved messages and native run output in one conversation. > - Reopening a chat rebuilt terminal transcripts and waited for network hydration. > - A queued scroll event could also disable bottom-follow after a cached editor increased the content height. > - This pull request reuses terminal history and preserves bottom-follow through that event order. > - The benefit is a faster cached return that opens at the latest reply. ## Linked Issues or Issue Description What happened? Returning to an agent chat could leave its latest reply below the viewport. Saved native transcripts also held the first reveal behind repeated network reads. Expected behavior A new chat navigation opens at the latest reply. A reader who scrolls up keeps their position. Back restores the earlier reading anchor. Saved history can render while the normal catch-up read runs. Steps to reproduce 1. Open an agent chat with saved native runs and task question cards. 2. Open Tasks, then reopen the same agent from Recents. 3. Observe the wait and the distance from the latest reply. Paperclip version or commit Reproduced on `523818ea9bf81f96abf4e6b73e81c2ce87518c07`. Rebased onto master `2f9d3d43c3`. Deployment mode Built from source with production UI assets and an isolated synthetic database. Related work: #18358 added recent-window chat loading. #18461 deferred completed transcripts for compact task opens. Open PR #14667 changes the reveal gate for task threads. This PR keeps that gate and caches hydrated terminal native history across visits. ## What Changed - Retain successful terminal native history in the application query cache. Keys include company, run, status and projection. - Render matching saved history immediately and continue the normal cursor-based catch-up read. - Preserve bottom-follow when content grows before a queued scroll event is delivered. Upward gestures and Back anchors retain priority. - Add regression coverage for remounts, catch-up, cache clearing, projection changes, cached question editors and scroll event ordering. - Repair two browser assertions: use a final 422 failure for explicit history retry, and accept saved task-view query parameters. - Document the local before/after comparison, including unchanged paths. ## Verification - Focused transcript and scroll suites: 51 tests passed after the review fixes, including a 300-run cache stability case. - Full UI suite on the rebased head: 896 files passed; 11,257 tests passed and one skipped. - Final review-fixed production UI: all 24 desktop/mobile chat journeys passed. - Final head: `pnpm -r typecheck`, `pnpm build` and `pnpm check:token-gates` passed. The complete GitHub CI test matrix, E2E, build and release checks passed for `4429a9aab4`. - The local repo-wide test command was stopped before performance sampling. Local acceptance used the full UI suite and the chat browser rig; CI completed the full repository matrix. - A parallel local build initially read a partially regenerated runner module while typecheck was emitting files. After typecheck finished, the serial full build passed. No runner source change was required. - The first browser run exposed the retry fixture's synthetic 500 being retried by the shared query policy. A final 422 response now verifies explicit retry. The Back assertion also now accepts the task collection's saved view query. - Re-measured the final reviewed build against the original baseline in five alternating pairs. The same retained 600-message/300-run fixture, Chromium at 1440×900 and 150 ms simulated latency produced cached-return medians of 580 ms (568–594) before and 155 ms (130–161) after. Median paired saving: 433 ms. All five pairs improved. A separate loopback asset host kept the manual test-drive build unchanged. - The initial full comparison found no reliable change to loopback cached returns, cold opens or uncached in-app entry. These are local results, not deployed staging measurements. - Greptile identified cache-expiry row loss and missing older output after a cached projection switch. Regression cases reproduce those paths. Mounted history now retains restored rows, and each projection retains its confirmed snapshot. The fresh review on commit `4429a9aab4` is 5/5 with no open findings. - One browser run overlapped a local rebuild and received a root-page 500. The server log records the missing generated index at that time. The run also timed out waiting for wheel acknowledgement after older rows had loaded; that input-acknowledgement cause was not established. After the build completed, the full serial run passed all 24 cases. - Run the opt-in browser rig with `tests/perf/agent-chat/playwright.config.ts` against a disposable loopback instance. See `tests/perf/agent-chat/README.md`. ## Risks - Cached history must not cross company, status or projection boundaries. Regression tests cover those cache misses and cache clearing. - Terminal output can receive late events. Cached history still performs catch-up reads, and retry, reconnect and mode changes retain their reads. - Programmatic scroll events can race with a reader gesture. Tests cover upward intent, delayed media and Back restoration. - This change does not improve cold-start latency. No schema, server authorization or exported data changes. ## Model Used OpenAI Codex using GPT-6. The runtime does not expose a more specific model ID or context-window size. Used reasoning, shell execution, repository editing, tests and browser profiling. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 09:38:40 -06:00
Dotta Paperclip Devin Foley	51a65af9b8	fix(recovery): continue preserved preparation after process loss (#19464 ) ## Thinking Path > - Paperclip manages AI agents and their execution state. > - A server restart can stop workspace preparation before provider execution. > - Workspace recovery must preserve files before the task can continue. > - Recovery handled interrupted preparations but skipped failed process-loss records with the same no-provider certificate. > - A withheld first retry could also hide the remaining retry budget. > - This change resumes certified preparations through normal admission after file recovery. ## Linked Issues or Issue Description What happened? A process-loss attempt remained failed after workspace recovery completed. Its first retry had been cancelled before dispatch by the workspace hold. The task had no running attempt or automatic continuation. Expected behavior Continue once after the server proves that provider work did not start, file preservation completed, and the exact leases stopped. Retain the existing failure budget and all task admission gates. Steps to reproduce 1. Stop a Runner preparation before native runtime selection. Record the restart recovery bootstrap certificate. 2. Retain the workspace and withhold the first automatic retry before dispatch. 3. Complete file preservation and restart the recovery worker. 4. Observe that the old code schedules no successor. The regression test reproduces this with PostgreSQL and real filesystem operations. Paperclip version or commit Reproduced against master at `2f35eb1080`. Related prior work: #19422 continues saved user input after verified copy-back. This change handles automatic continuation of certified process-loss preparation. ## What Changed - Accept failed process-loss preparation only with the durable no-provider bootstrap certificate and existing execution, lease and file checks. - Discover and settle the recovered source through the existing preparation recovery path. - Continue from a verified withheld retry without rewriting its history or resetting its failure count. Adopt it only as the sole retry child; multiple-child history reuses the newest existing child deterministically. - Exercise automatic dispatch, preserved-file readback, one provider turn, accepted result and task completion in the local native-session fixture. - Keep a durable continuation receipt after file-hold settlement until a successor is recorded. Known gates wait for existing eligibility events. Unresolved delivery keeps its shared recovery deadline. - Document the recovery and retry-budget rules. ## Verification - Red: both new PostgreSQL/filesystem regressions failed with zero scheduled successors where one was required. - Green: 54 focused recovery, automatic-dispatch, interruption, multiple-child history and admission safety cases pass on `13c27c2c2f`. The full PostgreSQL admission adapter suite passes all 83 tests; server source declaration typecheck also passes. Local environment: Darwin arm64, Node 26.4.0, PostgreSQL 18.1, Vitest 5.0.3. - Additional red/green: interruption after committed settlement, pause/resume, failed enqueue after cursor advancement, and absence of an earlier retry all failed before their repairs. Restart before enqueue, restart after the successor claim, and pause/resume now preserve one continuation. A rolled-back eligibility change cannot start execution. Deferred and delivered records have no retry deadline. - Local acceptance uses the real heartbeat dispatcher, isolated files and PostgreSQL, with a simulated native provider. It reads the preserved file, writes continued work, accepts the structured result and completes the task once. - Negative cases retain no-certificate, provider-evidence, live-owner, Stop, missing-stop-receipt, wrong-snapshot, human-block, pause, reassignment, approval and exhaustion fences. - CI fixture regression: 4 error-propagation/concurrent-resolution cases pass after adding the new obligation read to their transaction stubs; existing assertions remain unchanged. - Existing idle-drain and notification contract regressions: 4 tests pass after restricting recovery wakes to actual durable obligations. Late approvals block scheduled, queued and final dispatch. - Additional existing retry-scheduling suite: 62 cases pass locally. The pre-provider quota case exceeds its existing 5-second terminal wait on this loaded Darwin host, including when tested alone and with the preceding retry implementation. It remains a local timing limitation; the assertion and timeout were not changed. All Linux CI server shards pass on the combined head, including this unchanged suite. - Existing run-control, legacy-recovery and shared execution-control lifecycle suites: 128 tests pass with the review repair. Required CI supplies final-revision broad coverage. - Real-Codex acceptance was not run: automatic approval review rejected authenticated provider execution because it could incur cost/external effects and required explicit approval. No provider call ran; the controlled-provider acceptance covers the causal restart and durable scheduling behavior. - Hosted acceptance and recovery of existing tasks require rollout. They have not been claimed from the local provider fixture. - Final-head review: Greptile 5/5 on `13c27c2c2f`, no outstanding findings. Another contributor added the multiple-child history safety repair in `13c27c2c2f` while CI was running. The delta was inspected and the 54 focused recovery cases rerun successfully. Required CI passes on that exact combined head: https://github.com/paperclipai/paperclip/actions/runs/42241794918. All 55 successful checks include aggregate `ci / verify`, typecheck, build, canary, all server and E2E shards, and Runner package/Rust/static checks. The separate Paperclip Review service check remains queued waiting for its assigned agent; no validated assessment has been returned. ## Risks A malformed preparation certificate or a false no-execution receipt could permit duplicate work. The existing checks still reject provider execution records, native coordinators, live controllers, unsettled leases and unverified files. The pending continuation must survive a restart without polling known admission blockers. A withheld retry retains its original row and consumes its remaining failure budget. Normal admission checks current ownership, approvals, pause and budgets. ## Model Used OpenAI GPT-6 family via Codex. The exact model ID and context window are unavailable in session metadata. Used reasoning, code editing and tool execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal task id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing>	2026-11-06 09:36:46 -06:00
Nicky LeachandClaude Opus 5.5	83c6832ccb	fix(build): typecheck package, Playwright, and e2e test files in CI (#18839 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner, plugin, and Playwright tests verify how those agents run, connect, and appear in the board > - Most package tsconfigs exclude tests, and Vitest strips types, so a type error in those tests fails nothing > - The tests drifted to 803 real type errors, and some of those errors hid assertions that checked nothing > - paperclipai/paperclip#18739 and paperclipai/paperclip#18738 check the server test tree and four test projects, but not these files > - This pull request adds a required check for these test files, fixes every error, and fails CI if a test file is in no program > - The benefit is that a type error in any test file outside `server/src` now fails CI ## Linked Issues or Issue Description Refs #18739 Refs #18738 What happened? 517 test files were in no program that required CI checks. A type error in those files did not fail CI. Expected behavior CI fails on a type error in any test file. Steps to reproduce Add `export const value: number = "text";` to a test file under `packages/paperclip-runner/src` on master. CI stays green, because `tsconfig.json` excludes test files and Vitest strips types. Measured gap Each row is a program with its tests included, on one thread. File and error counts are for head `7d4b5e907a`. Memory peaks were measured before the two latest merges of master. \| Group \| Test files \| Real errors \| Peak \| \| --- \| ---: \| ---: \| ---: \| \| `paperclip-runner` Node source \| 290 \| 644 \| 1.09 GiB \| \| `paperclip-runner` surfaces \| 9 \| 12 \| 0.42 GiB \| \| `paperclip-runner` devtools and examples \| 14 \| 0 \| 0.68 GiB \| \| `sandbox-providers/kubernetes` \| 21 \| 65 \| 0.44 GiB \| \| 8 other sandbox providers and `bridge-template` \| 32 \| 4 \| 0.34 GiB \| \| `plugins/sdk` \| 22 \| 15 \| 0.41 GiB \| \| `plugin-workspace-diff` \| 5 \| 0 \| 0.37 GiB \| \| `plugin-orchestration-smoke-example` \| 1 \| 1 \| 0.15 GiB \| \| `kv-demo-mcp-server` \| 3 \| 0 \| 0.13 GiB \| \| `packages/db` scripts \| 1 \| 0 \| 1.01 GiB \| \| 8 Playwright suites under `tests/` \| 52 \| 25 \| 1.72 GiB \| \| `tests/runner-acceptance` \| 3 \| 0 \| 0.58 GiB \| \| `tests/e2e`, through layer declarations \| 63 \| 37 \| at most 1.59 GiB \| \| `ui` `AttentionInteractionResolver.test.tsx` \| 1 \| 0 \| n/a \| \| Total \| 517 \| 803 \| \| 27 of the 803 errors came from master while this pull request was open. Each merge of master brought new test code that no check had read. Errors that were config errors, not real errors: - One TS2688 for each provider, because the root install does not install provider dependencies. - 9 in the Runner Node program (TS7016 and TS6059). - 2 in `tsconfig.browser.json`. TypeScript 7 removed `baseUrl`, so that config was invalid. - 213 in the Playwright suites (`@/` aliases and Vite types). - About 60 in `tests/e2e` (import extensions and aliases). `packages/plugins/create-paperclip-plugin` has no gap. Its tsconfig excludes its test, but `cli/tsconfig.json` reaches that test through an import. `server/test-baselines` also has no gap, because `tests/lifecycle-baseline` reads it. ## What Changed - Added `scripts/typecheck-test-programs.mjs` and its tests. The `Typecheck + Release Registry` job runs it. It checks each program in `PROGRAMS` on one thread. - The script fails if a tracked `.test.ts(x)` or `.spec.ts(x)` file outside `server/src` is in no program. The compiler file list decides this, because `include` drops `foo.test.tsx` when `foo.test.ts` exists. - Added a `tsconfig.test.json` to each uncovered package, and `tests/tsconfig.playwright.json`. - Added `tests/e2e` to `PROJECTS` in `scripts/typecheck-test-projects.mjs`, with a `tsconfig.json` and a `tsconfig.layers.json`. - Added `scripts/typecheck-sandbox-providers`. It is a small workspace that installs provider SDKs from one lockfile. I did not add a lockfile inside each provider, because `scripts/build-standalone-public-packages.mjs` uses a frozen install when one exists, and that changes what a release resolves. - Corrected `packages/paperclip-runner/tsconfig.browser.json` for TypeScript 7. - Set `rootDir` to `..` in `packages/paperclip-runner/tsconfig.test.json`, because one Runner test imports from `packages/adapter-utils/src`. The program does not emit files, so this changes no build output. - Moved the Kubernetes provider from `vitest ^3.2.4` to `^4.1.8`, the version the other providers use. Vitest 3 requires `tinypool ^1.1.1`, which has two critical advisories (GHSA-85c8-ppgw-ccpr and GHSA-5gmw-xhrv-c9v3). The first patched version is 2.1.2. Its tests have the same result on both versions: 189 passed, and the same 12 `file-sync` tests failed. - Moved the Kubernetes provider from `zod ^3.24.2` to `^4.6.6`, and gave its two `z.record` calls a key schema. This is the only production change, and it stays in this pull request. See "Production change for review". - Renamed `AttentionInteractionResolver.test.ts` to `AttentionInteractionResolver.replace.test.ts`, so TypeScript reads the `.test.tsx` beside it. - Fixed all 803 errors, with changes to test files and test support only. The fix for `contracts/native-execution.test.ts` is the same as the fix in paperclipai/paperclip#18739, so those two changes merge cleanly. - Updated `doc/DEVELOPING.md`. This pull request does not edit `server/scripts/typecheck-layers.mjs`. paperclipai/paperclip#18739 and paperclipai/paperclip#18738 have both merged, and this branch includes them. ### Production change for review The new lockfile made dependency review flag `zod@3.25.76` as a high-severity advisory (GHSA-mrhx-c7xc-m3w5). The first patched version is 4.6.6, so 3.x has no fix. The published Kubernetes plugin uses zod to validate operator config. The zod 3 source does not compile against zod 4, so a typecheck-only override cannot fix this. The existing tests do not prove this change, because they also passed with the invalid one-argument `z.record`. So I parsed 45 inputs with the same schema on zod 3.25.76 and zod 4.6.7: \| Result \| Inputs \| Detail \| \| --- \| ---: \| --- \| \| Identical \| 23 \| Includes defaults, valid config, and a `__proto__` key \| \| Same outcome, new error wording \| 21 \| For example, `Invalid url` is now `Invalid URL`. `validateConfig` shows this text to operators. \| \| Different outcome \| 1 \| `jobTtlSecondsAfterFinished: 2 * 60` was valid and is now rejected. The new limit is `Number.MAX_SAFE_INTEGER`. \| I did not test this against a real cluster. The decision is to keep this change in this pull request. Since then, paperclipai/paperclip#17364 has moved the rest of the repository to zod 4.6.x, so the Kubernetes plugin now matches it. ### Behavior change for review `reducedMotion` under `use` is not a Playwright option. It had no effect in 11 configs and 2 `test.use` calls, so those suites ran with `no-preference`. The options are now under `contextOptions`, so reduced motion now takes effect. `app-shell-design.spec.ts` asserts a real transition, so it now asks for `no-preference` itself. The baseline manifest has `snapshotCount: 0`, so this invalidates no published baseline. A frozen trial reference PNG can differ. ### Findings These are product questions. I did not change them, and I did not hide them with casts: 1. `PrpEvent` and its schema disagree. The TypeScript type in `src/protocol/replay-contract.ts` requires `normalizedSessionId`. The `required` list in `protocol/schemas/event.schema.json` does not include it. `validatePrpEvent` accepts events that the type forbids. 2. `requestCheckboxConfirmation` is stricter than its runtime. It uses the output type, so plugin authors must pass `continuationPolicy` and `allowDeclineReason`, although the schema gives them defaults. `z.input` would fix this. 3. `RetainedRunnerdMaintenanceInput.backend.kind` is a free string. The quarantine domain comes from `descriptor.kind`, but the release uses `proof.backend.kind`. If they differ, `completeRetainedNativeSessionCleanup` returns 0 and the quarantine stays. The production caller passes `"runner"` today, so this is not a live bug. 4. `CodexHarnessSession.close(input?)` is optional, but the contract requires `{ reason }`. These are tests that verified nothing. I corrected them, and they pass: - `backends/native-instruction-measurement.test.ts` filtered on `placement === "core"`, which is not a valid placement. It measured an empty tool list. It now measures 14 tools. - `drivers/acpx/codex-acpx-driver.test.ts` checked for `usage.updated`, which is not an event type. - The `hostFixture()` helper in `drivers/acpx/runtime-host.test.ts` ignored two overrides, so `expect(prepareSandbox).not.toHaveBeenCalled()` could not fail. - `live/live-session.test.ts` set `agent: "pi"` before a spread that overwrote it. - `native-distribution-integrity.test.ts` passed an AbortSignal where `openCommand` expects `{ signal }`. These are gaps that this pull request does not close: - The root lockfile still has some old zod versions. paperclipai/paperclip#17364 moved the direct zod dependencies to 4.6.x. `pnpm-lock.yaml` still resolves `zod@3.25.76`, `4.1.11`, and `4.4.3`, which are below the patched 4.6.6. `packages/paperclip-runner/scripts/pi-distribution/package.json` pins `zod` to `3.25.76`, and some agent SDKs resolve `4.4.3`. Dependency review does not flag them, because this pull request does not add them. - Dead GitHub coverage. `chat-adapters-ui-providers.spec.ts` returned early for GitHub, so four later `provider === "github"` branches never ran. I removed them. They included the private key file and paste match, and the reconnect flow. Nothing asserts those behaviors in that spec now. - Provider tests do not run in CI. The root `vitest.config.ts` lists only Daytona. The other 8 providers, `bridge-template` included, are typechecked now, but CI does not run their tests. - One `packages/db` test never runs. `scripts/clean-poisoned-claude-sessions.test.ts` is outside the `include` of `packages/db/vitest.config.ts`. - `plugin-orchestration-smoke-example` does not collect. `@paperclipai/shared` is not linked in its `node_modules`. It passes 3 of 3 with an alias. ## Verification Run these commands: ```sh node scripts/typecheck-test-programs.mjs --low-memory pnpm --filter @paperclipai/server typecheck node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations node --test scripts/typecheck-test-programs.test.mjs ``` Results on macOS (arm64), Node 26.12. CI uses Node 24, and its `Typecheck + Release Registry` job also passes. At head `7d4b5e907a`, the test programs, `tests/e2e`, and the script tests pass again with 0 errors. The times and peaks below come from the run before the two latest merges of master: \| Check \| Result \| \| --- \| --- \| \| Test programs, `--low-memory` \| Exit 0, 0 errors, 42 seconds, peak 1.78 GiB \| \| Server layers, low memory \| Exit 0, 0 errors, 38 seconds, peak 3.41 GiB \| \| Test projects, including `tests/e2e` \| Exit 0, 0 errors, 13 seconds, peak 1.59 GiB \| \| Script tests \| 12 passed \| \| Injected type error in each of 23 places \| Each one failed its program with TS2322 \| \| Coverage check \| 0 unchecked files, 10 seconds \| I did not run this in a 4 GiB container. The peaks above are `maximum resident set size` for the largest single process. Runtime check of the fixes. Types do not prove that a test still works, so I ran the suites before the two latest merges of master: \| Suite \| Result \| \| --- \| --- \| \| Runner, full suite \| 4,885 passed, 3 failed, 13 skipped \| \| Sandbox providers, 10 packages \| 939 passed, 12 failed, 36 skipped \| \| Plugin SDK \| 200 passed \| \| `kv-demo-mcp-server` \| 12 passed \| \| Renamed UI test and its sibling \| 4 passed \| All 15 failures exist without these fixes or come from the local machine: - 3 Cursor Cloud timeouts. `artifact-delivery.test.ts` has 1 and `cursor-cloud-driver.test.ts` has 2. They time out in a full local run, and all 133 tests in those files with `codex-security-config.test.ts` pass when run alone. They fail in the same way with those two files at their unedited versions. - 12 Kubernetes `file-sync.test.ts` failures. They fail in the same way before and after the fixes. The first failure is that `/proc/self/fd` does not exist on macOS. After each of the two latest merges of master, I ran only the Runner test files that I changed: \| Merge \| Result \| \| --- \| --- \| \| `885c249cb9`, 8 files \| 534 passed \| \| `7d4b5e907a`, 7 files \| 672 passed, 16 failed \| The 16 failures are in the two `source-reconciliation` test files. They fail in the same way with master's unedited versions of those files. 15 of them fail with `ACPX sidecar exited at initialize`, so I think my local build of the Rust and sidecar binaries is out of date. I did not rebuild them to check. CI builds them from scratch, and all Runner jobs pass in CI at `7d4b5e907a`. Not run: - `tests/e2e` and the 8 Playwright suites need a server, so most of them have only compile checks. `playwright test --list` loads 308 tests from the main e2e config. - `packages/plugins/sandbox-providers/kubernetes/test/integration` needs a cluster, so it has only compile checks. - I did not run the full `pnpm test:run` or root `pnpm build`. Required CI runs them. Reduced motion. I ran 6 motion-sensitive Storybook specs in system Chrome with a temporary config, once with `reduce` and once with `no-preference`. Both runs had 45 passed and the same 5 failed. So reduced motion did not change those results. The 5 failures do not depend on motion. At least one comes from the browser, because system Chrome prints `oklch(55.6% 0 0)` where the test expects `oklch(0.556 0 0)`. I did not run the other Storybook specs. The `storybook-visual` label runs that workflow. ## Risks - A new lockfile to maintain. A change to a provider `package.json` now fails the typecheck job until someone refreshes `scripts/typecheck-sandbox-providers/pnpm-lock.yaml`. The failure message gives the command. - The job has a new install. The step downloads provider SDKs, so it depends on the npm registry. It uses `--frozen-lockfile` and `--ignore-scripts`. - The local install replaces provider `node_modules`. A contributor with a standalone provider install must reinstall that provider after running the script. - Reduced motion can change Storybook results. See "Behavior change for review". That workflow is not part of required CI. - Kubernetes plugin validation changed. A release of this plugin ships zod 4. Error wording changes, and very large integers are now rejected. See "Production change for review". - Master moves fast. Each merge of master brought new test code with type errors, 27 so far. After this merges, the check reports those errors in the pull request that adds them. ## Model Used - Provider: Anthropic Claude, through Claude Code - Model: Claude Opus 5.5 (`claude-opus-5-5`) - Capabilities: tool use, code execution, and subagents for the type fixes in separate files ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details. The app generated the branch name. - [ ] I have run tests locally and they pass. See the failures explained in Verification. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 07:33:59 -08:00
DottaandClaude Opus 5.5	2b2004bf50	test(runner-e2e): gate completion briefs inside the run's own task directory (#19475 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent Chat returns delegated-task completions to the chat that started them, and the runner E2E `completion-updates` suite checks this with real providers > - The restart cell (`completion-updates.runner-codex.local.handoff-completion-restart`) failed before its server restart with "Timed out waiting for worker waiting while originating chat is idle" > - Since project-optional task workspaces (#15749), a task in a project without a workspace runs in its isolated task directory. A native chat runs in its isolated conversation directory. Native Codex can read only that directory. > - The fixture still wrote its brief wait scripts into the managed project workspace and the agent home. Codex got EPERM on both scripts, wrote the note from the task record, and finished. The worker never reached the gate, so the setup poll timed out. > - This pull request makes the waits name only cwd-relative files and publishes each brief into the run's own isolated task directory > - The benefit is that the restart cell reaches its real boundary again (durable Done, source provider held at the gate, server restart) and tests restart recovery instead of a stale fixture path ## Linked Issues or Issue Description What happened? The runner E2E cell `completion-updates.runner-codex.local.handoff-completion-restart` timed out in setup. The run ledger shows both agents report `permission_denied` (EPERM) for `node .../wait-for-brief-.cjs`. The worker run cwd was `<instance>/isolated-workspaces/<company>/<issue>`. The fixture wrote the script to the managed project workspace and the agent home. What did you expect to happen?* The worker waits at the brief gate while the chat is idle. Then the cell releases the brief, holds the completion reply at its gate, and restarts the server. Steps to reproduce `pnpm test:e2e:runner -- --max-automatic-retries 0 --id completion-updates.runner-codex.local.handoff-completion-restart` on master `6cafcaf89b`. Classification Test defect. The failure happens before the restart. Workspace isolation is the intended product behavior. ## What Changed - `tests/runner-e2e/chat-stories.ts`: add `workspaceChatBrief(nonce, timeoutMs)`. It returns a bounded `node -e` command that reads and writes only cwd-relative files, plus `at(dir)` to find the gate and ready files in a workspace. - `tests/runner-e2e/completion-update-flow.ts`: the worker and completion instructions use that command. The fixture reads readiness from, and publishes briefs to, `isolated-workspaces/<company>/<issue>` for each delegated task and the source chat. The setup poll state now records the worker run cwd. Cleanup releases every wait that is still open. - `tests/runner-e2e/chat-stories.test.ts`: regression test. It runs the command from a temporary cwd, checks that the command names no host path, and checks the nonce and timeout bounds. It was red before the helper existed. - `tests/runner-e2e/catalog.ts`: bump the `completion-updates` definition to version 28 and set `workerBriefWorkspace: "run-isolated-task-directory"`. - `tests/runner-e2e/README.md`: describe the new brief location. ## Verification - `npx vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/chat-stories.test.ts tests/runner-e2e/completion-updates.test.ts tests/runner-e2e/catalog.test.ts`: 108 passed. - Live cell, native Codex `gpt-5.6-sol`, local runner: `pnpm test:e2e:runner -- --max-automatic-retries 0 --id completion-updates.runner-codex.local.handoff-completion-restart`. Results are in the Verification comment below. - Two earlier local attempts in this worktree failed in setup with `harness_test_failed`. The worktree did not have a built `paperclip-runnerd`. After `pnpm --filter @paperclipai/paperclip-runner build`, the cell passed. ## Risks - Low risk. The change is in test fixtures only. No product code changes. - The idle, busy, and multiple cells share this flow. They now also use the isolated task directory. That is the same directory the product selects for these tasks. - The agent must run the inline command from its cwd. The instructions say this, and the command fails closed (no gate) if the agent runs it elsewhere. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits). Default context window. Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 09:32:50 -06:00

1 2 3 4 5 ...