mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
4cf04d64cb07a201fdcef05928e49a463a8641a2
7933
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4cf04d64cb |
fix(native): retry a transient endpoint failure after a same-run bootstrap restart (#21488)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The native runtime starts a Runner in a remote sandbox for each agent run. A bounded same-run retry (at most 3 startup attempts) handles transient startup failures. > - A server restart can stop a run during its first Runner bootstrap. Restart recovery then claims `bootstrap_incomplete`. It stops and quarantines the unfinished bootstrap and launches a fresh bootstrap on the same run. > - The executor builds its "unclaimed bootstrap" proof only when `restartRecovery` is not set. Thus the fresh bootstrap of a `bootstrap_incomplete` restart can never prove that a transient failure is safe to retry. > - A transient HTTP 502 from the Runner endpoint request in that fresh bootstrap was then classified as `ambiguous_state`. The run stopped and needed manual board recovery. > - This pull request gives the unclaimed-bootstrap proof to a `bootstrap_incomplete` restart as well as to a first attempt. > - The benefit is that a transient endpoint failure after a restart uses the bounded automatic retry and does not need a person to recover the run. ## Linked Issues or Issue Description Refs #20414 (related: it fixes executor admission of a `bootstrap_incomplete` restart at state-root migration. It does not change the retry evidence that this PR changes.) **What happened?** A server restart interrupted the first Runner bootstrap of a remote native run. Restart recovery claimed `bootstrap_incomplete` and launched a fresh bootstrap on the same run. The fresh bootstrap registered its controller checkpoint. Then `runner.ingress.acquire` failed with HTTP 502, before any Runner connection or provider event. The run ended with `nativePhase: terminal_failure`, `recoveryOwner: board`, attempt 1 of 3, and the next action "resolve the ambiguous session state". The error was `Runner connection endpoint could not be acquired (HTTP 502).` (`runner_ingress_acquisition_failed`). **What did you expect to happen?** The transient 502 occurred before the Runner connected and before any provider work. It should use the bounded same-run startup retry, as it does on a first attempt. **Steps to reproduce** 1. Start a native run on a remote sandbox target. 2. Restart the server during the first Runner bootstrap, so that restart recovery claims `bootstrap_incomplete`. 3. Make the next `runner.ingress.acquire` call return HTTP 502. 4. See that the run stops as ambiguous state and does not schedule a startup retry. ## What Changed - Add `attemptLaunchesFreshBootstrap()` to `native-bootstrap-proof.ts`. It returns true for a first attempt and for a `bootstrap_incomplete` restart. It returns false for `reattach_existing_runner`, `reattach_remote_runner` and `resume_dead_runner`, because these continue an existing Runner or provider session. - In `native-session-executor.ts`, use this helper in place of `!input.restartRecovery` when the executor builds the unclaimed-bootstrap evidence for a startup failure. - Add `native-bootstrap-proof.test.ts` for the helper. - Add a case to `native-startup-retry.integration.test.ts`. It runs the real executor under a `bootstrap_incomplete` restart claim, and the fresh bootstrap gets a typed HTTP 502 from the Runner endpoint before the Runner connects. ## Verification - `native-bootstrap-proof.test.ts`: the test "accepts a same-run restart of an incomplete bootstrap" failed before the fix and passes after the fix (5/5 tests pass, vitest 5.1.0). - `native-startup-retry.integration.test.ts`, case "transient 502 after a bootstrap_incomplete restart": the run must settle as `retryable_failure` with `recoveryMode: "bootstrap_retry"`, attempt 1 and a set `nextAttemptAt`. Without the executor change, the executor builds no bootstrap evidence and the run settles as `ambiguous_state`. CI runs this suite; I could not run it locally. - The other guards for a retry did not change. The checkpoint must be unclaimed, there must be no provider events and no session checkpoint, and the 3-attempt startup budget applies. If the earlier attempt opened a provider session, provider events exist and the run still fails closed as ambiguous. - I could not run the full typecheck and the real-Runner integration suites locally (no Rust toolchain in the sandbox). CI must run them. ## Risks - Low risk. The change only affects a `bootstrap_incomplete` restart whose fresh bootstrap fails during startup. Such a failure now gets the same retry decision as a first attempt. All the other evidence checks still apply, so a run with provider activity still stops for manual recovery. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code with tool use (shell, file edit, and code execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
532e502791 |
fix(heartbeat): drain native resume dispatches whose timer already fired (#21479)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat service runs agent work and resumes interrupted native sessions on timers > - `drainActiveRunExecutions()` lets a caller that tears down shared state (a test cleanup) wait until no background run work is in flight > - The drain cleared resume timers that had not fired yet, but it could not see a timer that had already fired and was still reading and claiming run state > - That untracked work raced the `fresh-continuation-fallback.test.ts` cleanup TRUNCATE and deadlocked it, which failed Cloud readiness and held releases on 2026-11-13 > - This pull request makes the drain wait for fired resume dispatches too, and removes the deadlock retry from that suite's cleanup > - The benefit is a deterministic cleanup: nothing still writes run tables when the drain returns, so the release gate no longer fails on this test ## Linked Issues or Issue Description No public issue exists. The bug is described here. **What happened?** Cloud readiness `verify / General tests (server (5/10))` failed on master commits |
||
|
|
342fb308bd |
perf(run-events): commit streaming text deltas in batches (#21460)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One server process stores every event of every running native agent in `heartbeat_run_events` > - Text deltas (`item.delta`) are about 85% of that process's SQL statements: each delta is one transaction of four statements on the shared 10-connection pool > - At 50 agents and 20 deltas per second, the process falls behind and unrelated agent requests wait for the pool > - This pull request commits text deltas in batches: one transaction for all deltas that arrive within 250 ms, with one multi-row insert > - The benefit is about 7 times fewer transactions and 5–6 times fewer statements per delta, with no delay for the first text of an item ## Linked Issues or Issue Description Refs #21435 (this pull request is stacked on it). Refs #21131. Item 1a of the agent scaling plan in #21418. **What existing behavior does this improve?** The persistence of native text deltas. `native-session-runtime` awaited `PaperclipControlPlanePort.appendEvent` for each event, and the port ran one transaction per event. **Subsystem affected** Server run event ingest (`paperclip-control-plane-port.ts`, `heartbeat-run-events.ts`) and the runner package's turn consumer (`native-session-runtime.ts`). **Current behavior** Each delta takes one transaction: BEGIN, run lock, append, COMMIT. In the load model with 50 agents (deltas only), that is about 4.0 statements and 1.0 transaction per delta. **Proposed behavior** The turn consumer starts the next append without waiting for a delta's receipt. The port collects deltas and commits them in one transaction with one insert. In the same load model, that is about 0.7 statements and 0.14 transactions per delta. **Reason and benefit** Fewer transactions free the shared pool and the main thread for other agents' requests. This is the largest per-agent cost in the scaling plan. **Breaking changes** None. The row shape does not change. A port without the new capability keeps the earlier serial behavior. ## What Changed - `appendHeartbeatRunEventBatch` (`heartbeat-run-events.ts`) appends native source events of one run and one source in one transaction. When the events continue the stored source cursor (from #21435), it sends four statements for the whole batch: the run lock, one replay probe, one `insert ... select from jsonb_array_elements(...)` of every row, and the commit. Any other batch (a replay, a gap, an event with a wake, or a run without a cursor) appends its events one by one in the same transaction, with the single-append checks. - **Storage choice: one row per delta, written by one statement per batch.** I did not store one row per batch. The transcript, live events, run log export, chat progress, replay (`replayEvents`), recovery cursors and the UI all read one row per delta. One statement per batch keeps every reader unchanged, and rows written before this change read the same. - `PaperclipControlPlanePort` sets `pipelinesStreamingEvents`. An `item.delta` appended with the new `streaming` option goes into a queue: - It commits at once until a delta with text commits after the latest other event. So the first text of each item, and the time to first token, never wait for the timer. - Later deltas commit after at most 250 ms, or at once when the batch has 100 deltas. - Every other event, `checkpointSession`, `completeRun`, `replayEvents` and `provesNoRunProviderWork` first commit the queue. A non-streaming event is never committed before the deltas that preceded it. - A delta appended without `streaming` commits at once, so callers that wait for each receipt are not slowed. - The receipt of a delta settles only after its batch commits and its observers (live event, executor callback) run, in order. - If a batch fails, every delta in it rejects and nothing from it is stored. - `native-session-runtime` `consumeTurn` appends deltas with `streaming` and does not wait for their receipts (at most 256 pending). Before any other event, any result, and the governed-wait settlement, it calls `flushStreamingEvents()` and then waits for all pending receipts, so a boundary never waits for the timer. A failed delta stops the turn before a later event is appended. The consumer joins every pending delta append before it settles, also when the provider stream fails. - If one observer of a committed batch fails, only that delta's receipt rejects. The other rows are still published and observed. - `ControlPlanePort` contract: optional `pipelinesStreamingEvents`, `flushStreamingEvents()` and `AppendControlPlaneEventOptions.streaming`. ## Verification - Red/green. The new tests failed before the change and pass after it: - Runner: "starts text delta appends without waiting and flushes them before any other event" (`native-session-runtime.test.ts`). The port in the test has no timer, so the turn finishes only if the boundary flushes. Before: the appends were serial, and later the boundary waited for the timer. - Runner: "joins pending delta appends before a failed turn settles". Before: the turn settled while a delta commit still ran. - Port: "observes every committed delta of a batch when one observer fails". - Port: "commits the first text delta at once and later deltas in one batch before the next other event". It checks one transaction and one insert for five deltas, flush before a non-streaming event, receipts only after commit, and immediate commit of the first delta with text. Before: each delta committed alone. - Port: "flushes queued deltas on the timer and before a replay read, and rejects every delta of a failed batch". It checks the crash-before-commit case: a revoked writer fails the batch, nothing is stored, and the resent deltas commit idempotently (the stored one is a duplicate). - Store: four new tests in `heartbeat-run-event-atomic-append.test.ts`: one lock, one probe and one insert per batch; replay idempotency and payload conflict; a batch that fails before commit stores nothing and its resend commits; the one-by-one path for no cursor and for gaps. - Focused suites pass: `heartbeat-run-event-atomic-append`, `heartbeat-run-event-sequencing`, `paperclip-control-plane-port` (including the package conformance suite), `paperclip-semantic-conformance`, `native-observer-replay`, `native-event-boundary-golden`, `live-recovery-work`, `native-interaction-bridge`, `chat-native-progress-work`, and `native-session-runtime.test.ts` in the runner package. - `pnpm --filter @paperclipai/paperclip-runner run typecheck:typescript` and `pnpm --filter @paperclipai/server run typecheck` pass. `pnpm check:module-boundaries` passes. - Load model (`scripts/benchmark-agent-scaling-load.ts` from #21418, `AGENTS=50 SECONDS=20`, local embedded PostgreSQL). The load model calls the store directly, so for this run I added a local `BATCH_MS` mode that applies the port's batching rules (first delta at once, 250 ms timer, 100 cap, flush before other events) and does not wait for delta receipts. That mode is not in this pull request. The machine was very busy (load average 40–60), so absolute figures are noisy; per-delta ratios are the stable result: | Deltas only | Statements/s | Transactions/s | Deltas done of offered | Statements per delta | Transactions per delta | Lateness p50 / p99 | | --- | --- | --- | --- | --- | --- | --- | | One transaction per delta | 2,920 | 730 | 14,668 of 20,000 | 3.98 | 1.00 | 3,074 / 5,384 ms | | Batched (250 ms) | 704 | 141 | 19,940 of 20,000 | 0.71 | 0.14 | 668 / 2,858 ms | | One transaction per delta (repeat) | 1,712 | 428 | 8,575 of 20,000 | 3.99 | 1.00 | 6,558 / 11,464 ms | | Batched (250 ms, repeat) | 856 | 172 | 20,000 of 20,000 | 0.86 | 0.17 | 187 / 556 ms | With all sources on, the machine was saturated (event-loop utilization 0.96–0.98) in both modes and the results were not usable (764 and 12,800 of 20,000 deltas done). ## Risks - **Ack and crash (read first).** In the live runnerd path the runner's ack does not wait for the `heartbeat_run_events` row today. `DurablePrpControlPlane.#event` acks after the in-memory journal commit. Streaming events are not in the controller checkpoint. So a delta that is acked but not yet appended is lost if the server crashes, and the runner does not resend it. This is the earlier behavior. This pull request makes that window up to 250 ms longer. The final text still survives through `item.completed`, which commits immediately and in order. This pull request does not change `durable-prp-control-plane.ts`. - Live text in the UI arrives in steps of up to 250 ms after the first text of an item. - A delta failure now surfaces at the next non-delta event or at the end of the turn, not at the delta. The turn stops before it appends a later event. - The governed-wait drain and the control-event replay still append one event at a time. They do not carry deltas. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0aaab644fe |
refactor(runner): remove the direct assigned-gateway path (#21311)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - An agent reaches the tools of its app connections through an MCP gateway. > - Before #21274, some native harnesses got the gateway URL and a bearer token and connected to the gateway directly. The Runner has code for that: an HTTP MCP server for ACPX providers, a server entry in the Codex `config.toml`, a server entry in the OpenCode config, and the lists that pass the three `PAPERCLIP_NATIVE_MCP_*` variables to a sidecar. > - #21274 gives each native harness its gateway tools through the PRP relay. Core removes the three variables from each Runner environment. The direct code path gets no binding. > - Code that cannot run still needs maintenance, and it can still put a token in a provider process if a variable arrives by another way. > - This pull request deletes the direct path from the Runner. > - The benefit is one path for assigned gateway tools, and no Runner code that can give a gateway token to a provider. ## Linked Issues or Issue Description No public issue exists. The issue is described here. This pull request is in a stack. Its base branch is `fix/keep-quiet-claude-provider-between-turns` (#21293). It depends on #21274, which is the bottom of the stack. On `master` without #21274, the direct path is still in use. **What existing behavior does this improve?** The Runner code that connects a provider directly to the assigned MCP gateway. **Subsystem affected** Paperclip Runner: the TypeScript drivers and the Rust `runner-core` environment lists. **Current behavior** After #21274, Core never gives `PAPERCLIP_NATIVE_MCP_NAME`, `PAPERCLIP_NATIVE_MCP_URL` or `PAPERCLIP_NATIVE_MCP_TOKEN` to a Runner. The Runner still has the code that reads them and configures a direct gateway server for ACPX providers, Codex and OpenCode. **Proposed behavior** The Runner has no code that reads the three variables to configure a provider. If the variables are in the environment of a Runner, no provider environment, sidecar, Codex `config.toml` or OpenCode config gets them. **Reason and benefit** Less code, one path, and no way for a gateway token to reach a provider process through the Runner. ## What Changed - Deleted `packages/paperclip-runner/src/drivers/native-mcp.ts` (`nativeMcpLaunchBinding`). - `drivers/acpx/runtime-host.ts`: removed the native gateway server entry and the name conflict check. The admission gate keeps its message. A run with a binding id is admitted when the sidecar uses the PRP transport with its tool bridge, as the production sidecar always does. - `drivers/acpx/environment.ts`: the name and the URL are not in the provider allowlist. - `drivers/runtime-context-materializer.ts` and `live/runnerd-codex-transport.ts`: the Codex home functions have no `nativeMcp` parameter. The sidecar token pass-through is gone. - `drivers/opencode/opencode-server-driver.ts`: removed the read of the binding, its trace redaction value, and the gateway server entry in the OpenCode config. - `drivers/stdio-mcp.ts` and `runner-core/src/stdio_mcp.rs`: removed the reservation of the gateway server name. No gateway server exists, so no name can conflict. The names `paperclip` and `paperclip_runner_*` stay reserved. - `runner-core/src/acpx_sidecar_transport.rs` and `codex_provider.rs`: the three variables are not in the pass-through lists. - `control-plane/durable-prp-control-plane.ts`: the three variables are not in the platform environment list. - `packages/paperclip-runner/README.md`: the text about direct callers and assigned servers. - Kept: the three names stay in the reserved lists (`src/configured-environment.ts`, `runner-core/src/configured_environment.rs`). A user cannot configure them. Tests in both languages now check all three names. - No server code changed. The relay needs the gateway token and URL on the control plane. 23 files, 73 lines added and 211 lines removed. ## State saved by the previous release - This pull request changes no persisted shape. The provider server lists, the launch profile digests, the protocol versions and the sidecar contract are the same as with #21274. - A Runner or sidecar that the previous release started keeps its own code until its run ends, as #21274 describes. - A kept Codex home that has the old gateway server in `config.toml` is written again at the next start and at a recovery of the same run. A test covers this. - If the old variables reach a new Runner, no provider gets them. Tests cover this for ACPX, Codex and OpenCode. ## Verification From `packages/paperclip-runner`: - `npx vitest run src/drivers/acpx/runtime-host.test.ts src/live/runnerd-codex-transport.test.ts src/drivers/acpx/environment.test.ts src/drivers/opencode/opencode-server-driver.test.ts src/live/runnerd-codex-transport-warm-attachment.test.ts src/drivers/runtime-context-materializer.test.ts src/drivers/agent-observability.test.ts src/drivers/stdio-mcp.test.ts src/control-plane/durable-prp-control-plane.test.ts src/configured-environment.test.ts`: `10 passed (10)` files, `807 passed` tests. - `npx tsc --noEmit -p tsconfig.json` and `-p tsconfig.test.json`: no errors. - `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no changes. - `cargo test -p paperclip-runner-core --test acpx_sidecar_transport`: 20 passed. `cargo test -p paperclip-runner-core --lib`: 639 passed, 4 ignored. Tests that failed with the old code and pass now: - `runtime-host.test.ts`: "admits a relayed … run with a binding id and ignores a previous-release gateway binding", for Claude, Codex and Cursor. - `runnerd-codex-transport.test.ts`: "drops a previous-release gateway binding from the … runner environment", for ACPX Claude, Codex, Pi, Cursor, Copilot and Kimi, and for OpenCode. - `opencode-server-driver.test.ts`: the OpenCode config has only the task bridge. - `runnerd-codex-transport-warm-attachment.test.ts`: a restored Codex home has no gateway server and no gateway credential in `config.toml`. - `environment.test.ts`: a provider environment has none of the three variables. - Rust `previous_release_gateway_binding_and_unrelated_secrets_never_reach_the_sidecar`: a Runner that starts with the old variables gives none of them to the sidecar, for five agents. Mutation check: with the three names put back in the Rust pass-through list, the Rust test fails. Not run: a live run with a real provider. Not run: `pnpm verify` and the full suite. CI supplies that coverage. ## Risks - **This pull request is safe only after #21274.** The stack order keeps that. - **A developer who sets the three variables by hand for `eval-session`** no longer gets direct gateway tools. No document describes that use. - **The warm attachment test uses the staged Runner binary from preflight.** That binary does not have the Rust changes of this pull request. The Codex path of that test does not use the changed lists. - The evidence that no entry point supplies a binding comes from a read of the code: Core removes the variables before it builds each Runner environment, and a native run cannot use the in-process path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change, reviewed the diff, and ran the verification and a mutation check again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), traced each code path and wrote the change and the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
12aa08721c |
feat(ssh): use the agent AI account in new environments (#21248)
Use the agent-selected AI account for new SSH environments. Preserve
remote-machine authentication for existing environments and expose it as
an advanced option. Keep managed credentials private while preserving
supported conversation state and credential refreshes across turns.
Validated with focused route, adapter, lifecycle, and real-process tests;
final CI passed on
canary/v2026.1113.0-canary.12
|
||
|
|
2a8b485140 |
fix(runner): accept object fields sent as JSON text and name wrong-type rejections (#21485)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip runner checks each tool call from the model against the tool's input schema before it sends the call to the control plane > - Some model hosts send an object argument as its JSON text. On the DigitalOcean host, `paperclip_finish` arrived with `continuation` as a string > - The runner rejected these calls, and the repeated-rejection guard then told the model that its arguments "did not arrive", which is wrong when every field arrived with the wrong type > - This pull request makes the runner parse such a string back into its object when that is safe, records each coercion as a run event, and makes the guard name the actual problem > - The benefit is fewer stuck finish calls, and guidance that points the model at the real fault ## Linked Issues or Issue Description Refs #21398 **What happened?** Chat-eval cohort 13 ran OpenCode with `openrouter/deepseek/deepseek-v4.1-flash-1017` on master `df1b12ad8e`. On the DigitalOcean host, 6 of the 8 remaining "Invalid tool input" rejections were `paperclip_finish` calls with `continuation` sent as a JSON string (`/continuation must be object,null`). Examples: `gateways-profiles.rotate-token`, `instance-operations.restart-service`, and `questions-and-approvals.multi-part-question`. The run diagnostic recorded `continuationShape: "string"`. After the second identical rejection, the guard from #21398 said "If you meant to send other arguments, they did not arrive". Every argument had arrived; one had the wrong type. **Expected behavior** When a schema field is object-typed, does not allow text, and the text parses to a plain JSON object that makes the call valid, the runner accepts the call with the parsed object and logs a run event. When a call stays invalid, the guard says which required fields are missing and which fields have the wrong type. **Steps to reproduce** 1. Start the OpenCode MCP bridge. 2. Call `paperclip_finish` with `reportedWorkDisposition: "yielded"` and `continuation` as `JSON.stringify({ kind: "response_wake", ... })`. 3. Before this change, the bridge returns `Invalid tool input: /continuation must be object,null`. ## What Changed - Added `packages/paperclip-runner/src/drivers/tool-input-coercion.ts`. `admitToolInput` validates the arguments. If they are invalid, it parses a string in an object-typed field back into its object. It does this only when the field's schema declares `object`, does not allow `string`, the text parses to a plain JSON object, and the whole coerced call validates. It applies to any object-typed field at any depth, not only `continuation`. - The OpenCode MCP bridge and the runner tool bridge use `admitToolInput`. They dispatch the coerced arguments, and they send one `paperclip.runner.tool-input-coerced/v1` diagnostic per accepted coercion. The OpenCode driver already records bridge diagnostics as run events. The diagnostic names the tool and the field paths. It never contains the values. The runner tool bridge has a new optional `onDiagnostic` option. - When parsed text is still invalid, the rejection describes what is wrong inside the parsed object, for example `/options/mode must be string`. - The repeated-rejection guard now says "Required fields X did not arrive" only for missing required fields. For fields with the wrong type, it says "Every field you sent arrived, but one has the wrong type: continuation must be an object or null, not text". ## Verification - `cd packages/paperclip-runner && npx vitest run src/drivers/opencode/mcp-bridge.test.ts src/drivers/runner-tool-bridge.test.ts` (63 tests pass). - Red/green: with the source changes reverted and the new tests kept, 5 tests fail: coercion acceptance and its run event (both bridges), no coercion for text fields or non-object text, the wrong-type guard wording, and the missing-field guard wording. With the change, all of them pass. - `npx tsc --noEmit -p packages/paperclip-runner` passes. - A narrow paid run of `questions-and-approvals.multi-part-question`, `runs-adapters.inspect-failed-run` and `budgets-costs.explain-hard-stop` ran on a branch that combines this PR with the two sibling cohort-13 PRs. The results are in a PR comment. ## Risks - Low risk. Coercion runs only after a call fails validation, and the call is accepted only if the coerced form validates. A field that allows text is never parsed, so a valid string never changes meaning. - Previous-release story (upgrade-safe run state): this change adds no persisted run, session, launch or pack state. It changes only how one in-flight tool call is admitted. Saved state carries forward unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2756be2f32 |
fix(models): answer an unreachable OpenRouter catalog with 503 instead of a crash report (#21468)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent setup lists the models that an OpenRouter key can select. The server gets this list from the public OpenRouter catalog. > - The catalog request has its own 10 second deadline. When the deadline expired, fetch rejected with a raw `TimeoutError` (a `DOMException`). > - The error handler answered 500 "Internal server error" and sent a crash report with no first-party stack frames. The user did not see the existing retry guidance. > - A slow or unavailable third-party catalog is not a fault in the Paperclip server. It must not create a crash report. > - This pull request maps catalog outages to a 503 response with the existing retry message and a stable code. Only those errors skip crash reporting. > - The benefit: users see "Retry or enter a model ID manually", and Sentry does not get noise from a provider outage. ## Linked Issues or Issue Description **What happened?** A staging server answered `GET /api/companies/:companyId/adapters/opencode_local/models` with `500 — The operation was aborted due to timeout`. At the same time, other outbound requests from the host were slow. Sentry recorded `TimeoutError: The operation was aborted due to timeout`. The event had only Node internal frames (`AbortSignal.timeout`). **Expected behavior** The route returns a clear "catalog unavailable" response. The UI shows the retry guidance. A third-party outage does not create a crash report. **Steps to reproduce** 1. Make `https://openrouter.ai/api/v1/models` slow (no answer for more than 10 seconds), or make it unreachable. 2. Request the OpenCode model list with `provider=openrouter`. 3. See a 500 response and a Sentry `TimeoutError` event. **Paperclip version or commit** `e23549cd9c631e4595a3213e03d0c960360452c6` I searched for duplicate PRs. #17083, #17961 and #10290 change the OpenRouter catalog contents or model validation. They do not change how the server handles a catalog timeout. ## What Changed - `server/src/services/openrouter-models.ts`: The catalog now maps three failure types to `HttpError(503)` with code `openrouter_catalog_unavailable` and the existing retry message: its own deadline, a classified transport failure (uses `classifyRemoteConnectionError`), and a 408, 429 or 5xx status. A module-local `WeakSet` records only the errors that it creates for these outages. `isOpenRouterCatalogUnavailable()` reads that record. - Other non-OK statuses become a 502 with code `openrouter_catalog_error`. These are still reported. Malformed JSON and an invalid catalog shape are also still reported. - `server/src/middleware/error-handler.ts`: The handler does not send a crash report for an error from `isOpenRouterCatalogUnavailable()`. It still answers 503 with the message and code. The new check is next to the existing MCP connection-failure exclusion. - Tests: unit tests for the deadline, DNS, 429 and 5xx cases, and for the failures that stay reportable. A route-level test sends a stalled catalog request through the real error handler. It checks for a 503 response and no crash report. It also checks that a different 503 is still reported. ## Verification - `vitest run server/src/services/openrouter-models.test.ts`: 7 tests pass. I ran this file alone because the full workspace install did not fit in the agent sandbox disk. - Red check: with the original `openrouter-models.ts`, 5 of the 7 tests fail. The deadline test receives `DOMException TimeoutError`, which is the same error as the production event. - `server/src/__tests__/openrouter-catalog-error-handler.test.ts` needs the full workspace dependencies. CI runs it. ## Risks - Low risk. Successful responses do not change. The cache and in-flight deduplication do not change. - The failure status changes from 500 to 503 (or 502) for the three routes that read this catalog: adapter models, fast-response models and prompt-injection-prevention models. The UI already shows `error.message`, so users now see the retry guidance in place of "Internal server error". - The crash-report exclusion is narrow. It applies only to error objects that this module created after an actual outage. A copied message or code does not match. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in the Claude Code agent harness with tool use (shell, file edits, Sentry, Railway and GitHub connectors). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0ebe6d846d |
perf(runner): keep a quiet Claude provider between turns of a run (#21293)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude Code through the ACPX sidecar. The Runner stops and starts the sidecar and Claude at each turn boundary. > - The reason was that a late tool call from an earlier turn could run in a later turn. #21278 removes that reason for Claude: the sidecar refuses a call that the active turn did not announce. > - A second reason remains. Claude Code works by itself between prompts. When a background task finishes, Claude starts a cycle of its own. If the process stayed alive with a live background task, that cycle could start between turns or mix with the next turn. > - So the Runner can keep the process only when it knows that Claude has no background work. > - This pull request adds that knowledge and uses it. After a Claude turn that completed with no background work, the Runner keeps the process for the next turn of the same run. > - The benefit is that a later turn of the same run starts with no provider restart. This is also the base for the next step, which lets a run continue when background work finishes. ## Linked Issues or Issue Description No public issue exists. The issue is described here. This pull request is stacked on #21278. Its base branch is `fix/bind-claude-tool-calls-to-turn`. Refs #19912 (design), #12419 (restart at turn start), #20145 (restart at park) and #20190 (restart at turn end). **What existing behavior does this improve?** The provider restart between turns of the same run, for Claude through ACPX. **Subsystem affected** Paperclip Runner: the ACPX sidecar (TypeScript) and the ACPX provider session in `runner-core` (Rust). Claude only. **Current behavior** - When a completed turn settles, the Runner restarts the sidecar and Claude (`restart_session_after_turn`, #20190). - If that restart did not run, the next turn restarts them at its start (`start_turn`). - The Runner does not know if Claude has background tasks. **Proposed behavior** - The sidecar asks `claude-agent-acp` for its async task reports. This is an optional capability of the pinned `claude-agent-acp` 0.73.0. The patch does not change. - A tracker reads those reports and the id of each background Bash command. It reports `{ live: 0 }` only when it knows that this process never had a background task and that Claude did not act between prompts. In all other cases the state is unknown. - The sidecar adds `backgroundTasks` to the terminal event of a completed Claude turn, and to `session.snapshot`. - The Runner keeps the process between two turns only when all of these are true: - the agent is Claude; - the turn completed (not failed, cancelled or interrupted); - the terminal event reported `backgroundTasks: { "live": 0 }`; - each existing condition of the turn-end restart is true; - before the next turn, a new `session.snapshot` reports the same identity and run, no active turn, no pending work, and `{ "live": 0 }`; - the settled-turn ledger and the tool replay ledgers are not full. - In each other case the Runner restarts as before. - A kept process keeps its tool call-id tombstones and its settled turn identities. **Reason and benefit** - A turn that follows a quiet turn in the same run starts without a restart. - The decision is safe by default: unknown state means a restart. ## What Changed TypeScript (`packages/paperclip-runner/src`): - `drivers/acpx/claude-background-tasks.ts` (new): `ClaudeBackgroundTaskTracker`. It reads raw ACP messages. It reports unknown when the capability is not negotiated, when a task finished, when a live task is not a shell command, when an update is malformed, when a limit is reached, when Claude acts with no client request in flight, or when Claude calls `CronCreate` or `ScheduleWakeup`. - `drivers/acpx/provider-failure.ts` and `acpx-runtime-adapter.ts`: Claude sessions also advertise the `asyncTasks` capability. The raw ACP observer gives each message to the tracker. ACPX 0.22 drops the `async_task_*` updates before the turn event stream, so only the raw observer sees them. - `drivers/acpx/runtime-host.ts`: `backgroundTaskState()`. - `cli/acpx-runtime-sidecar.ts`: the terminal event of a completed Claude turn carries `backgroundTasks` in the same payload as `goalAtTerminal`, with the same size limit. `session.snapshot` carries it when no turn is active. - `drivers/acpx/claude-tool-use-admission.ts`: `hasRoomForAnotherTurn()`. When the admitted-id ledger cannot hold one more full turn, the sidecar reports unknown, so the Runner restarts before the ledger is full. Rust (`packages/paperclip-runner/runner/crates/runner-core`): - `acpx_provider_session.rs`: the session records a quiet turn from the terminal event. `start_turn` continues in the same process only under the conditions above, and then uses `prepare_turn`, which keeps the tombstones. - `acpx_provider_generation.rs`: `can_continue_after_turn`, `continue_after_quiet_turn`, and the snapshot check `background_work_is_quiet`. - `acpx_park_preparation.rs`: the turn-end restart does not run for a quiet Claude turn. - `acpx_provider_backend.rs`: `session.reconciled` reports a replaced process only when the turn start replaced it. - `bin/fake-acpx-sidecar.rs`: new option `--background-tasks` for tests. - `runner/DURABLE_TRANSPORT.md`: the rule for a kept sidecar. Contract (`packages/paperclip-runner/protocol`): - The `backgroundTasks` report is `$defs.backgroundTasks` in `provider-schemas/acpx-sidecar.schema.json`: an object with only `live`, a non-negative integer. The schema validates it on `runtime.turn_terminal`. - `scripts/generate-acpx-sidecar-contract.mjs` generates the TypeScript type and reader and the Rust reader from it. The tracker, the sidecar and the Rust session use the generated code. - `fixtures/provider-boundary/acpx-background-tasks.json` has shared test documents. The TypeScript reader, the Rust reader and the schema are tested with the same accepted and refused shapes. - This changes the contract digest of the provider pack (`prp.acpx-sidecar` and `rust`). Older image packs are not reused. No provider `commandDigest` and no profile version changes. Not changed: park and run attachment. A process that ran a turn is still restarted at park, and only a process that ran no turn is bound to the next run. Other ACPX agents and Codex do not change. The `claude-agent-acp` patch does not change. ## State saved by the previous release - This pull request adds no persisted state. The three new fields of the session are in memory only. - A terminal event or a snapshot without `backgroundTasks` means unknown. An older sidecar therefore gets the restart, as before. - An older Runner with a new pack ignores the field and restarts at each turn, as before. - After a Runner restart, the in-memory state is gone. The next turn restarts the provider, as before. A test covers this. ## Verification From `packages/paperclip-runner`: - `npx vitest run src/drivers/acpx/claude-background-tasks.test.ts src/drivers/acpx/acpx-runtime-adapter.claude-background-tasks.test.ts src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts src/drivers/acpx/claude-tool-use-admission.test.ts src/cli/acpx-runtime-sidecar.test.ts src/drivers/acpx/acpx-runtime-adapter.test.ts src/drivers/acpx/runtime-host.test.ts`: `7 passed (7)` files, `363 passed` tests. - `npx tsc --noEmit -p tsconfig.json` and `npx tsc --noEmit -p tsconfig.test.json`: no errors. - `pnpm run typecheck:typescript` (it includes the contract checks): no errors. `node --test test/acpx-sidecar-contract.test.mjs`: 17 passed. - `cargo fmt --manifest-path runner/Cargo.toml --all -- --check`: no changes. - `cargo test -p paperclip-runner-core --test acpx_quiet_turn_continuation --test acpx_park_preparation --test acpx_park_readiness --test acpx_provider_turns --test acpx_provider_session --test acpx_warm_attachment --test native_provider_backend`: 4, 33, 8, 31, 12, 6 and 47 passed. No test failed. - `cargo test -p paperclip-runner-core --lib`: 637 passed, 4 ignored. - Mutation check: when the Rust code accepts each live count as quiet, two tests in `acpx_park_preparation` fail. New tests: - **Real ACP runtime** (`acpx-runtime-adapter.claude-background-tasks.test.ts`), with the pinned `claude-agent-acp`, the pinned Claude Code client and a local fake Anthropic API. A foreground Bash turn reports `{ live: 0 }`. A background Bash in the same process reports `{ live: 1 }` at the terminal event, and unknown after Claude ran its own cycle for the finished task. Each turn has its own usage receipt. One process serves all turns. - **Tracker unit tests** (13): spawn, completion, failure, stop, pause, a task that is not a shell, a subagent, no negotiation, malformed updates, activity between prompts, scheduling tools, session load replay, limits. - **Sidecar tests**: the terminal event carries the state for Claude and not for Codex. A full ledger reports unknown. - **Rust** `tests/acpx_quiet_turn_continuation.rs` (4) and `tests/support/acpx_quiet_turn_restart.rs` (7): - a quiet Claude turn continues in the same process, and a replay of a settled call id is refused; - live, unknown and malformed reports restart; - background work that appears after the terminal event restarts at turn start; - failed, cancelled and interrupted turns restart; - Codex with a quiet report restarts; - an active goal restarts; - 4,096 kept turns, then the next turn restarts; - park after a kept process still prepares a new process, and the next run attaches to it; - a Runner restart after a kept process resumes in a new process. - The existing restart tests did not change. The fake sidecar sends no report by default, so they are the tests for the unknown case. Live run with a real Claude account, not in CI: the real ACP runtime (ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code client), a real tool bridge, and the real Anthropic API with model `claude-sonnet-5-5`, on the code at the top of this stack. A turn that started one background shell reported `{ live: 1 }`. A turn that started two reported `{ live: 2 }`, then `{ live: 1 }` and `{ live: 0 }` as each shell finished. A turn in which Claude waited for its background shell in the foreground reported the unknown state. Not run: server integration tests (the fake sidecar there sends no report, so their process counts do not change), Linux, and a live run through the real sidecar process and runnerd. ## Risks - **All Claude sessions now negotiate the `asyncTasks` capability.** `claude-agent-acp` then sends `async_task_*` updates and adds a marker to a background Bash update. ACPX drops the updates. No other behavior of `claude-agent-acp` changes, from a read of its code. - **Park after a kept process pays the restart again**, as it did before #20190. With real Claude that restart was about 1.5 s. - **A small window remains** between the snapshot and the prompt of the next turn. Claude can start a cycle of its own in that window only without a background task, because a process that had a background task is not kept. - **Activity between prompts is found only when no client request is in flight.** A request that never gets a response would hide such activity. - **Background watchers cannot be told apart from other tasks**, so each task that is not a shell command makes the state unknown. - **Each kept turn adds one `session.snapshot` request.** - A multi-turn run with real Claude now uses one provider process. This path has tests with the fake sidecar and with the real ACP runtime. It has no test with the real sidecar host. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change, reviewed the diff, and ran the TypeScript and Rust verification again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), investigated the signals and wrote the change and the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
1a96247480 |
fix(recovery): continue a failed continuation again instead of escalating it to the board (#21394)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Automatic task recovery continues a task when its run ends for a
reason that is not a person's to fix
> - A continuation or recovery run can itself fail before it does any
work: a sandbox create timeout, a full sandbox disk, a lease mismatch, a
provider pack check, or a database deadlock
> - Today the stranded reconciler treats such a failure as non-retryable
and blocks the task for the board. The assigned-work sweep then leaves
the board hold alone, because its list of temporary causes is a short
allow-list. The held backlog pass uses a similar allow-list. Nothing
continues the task, and it shows "failed, Retry"
> - This pull request makes the cause test "not owned by a person" on
these paths, and routes a failed continuation run back through the
fresh-continuation fallback
> - The benefit is that such a task continues from its saved work, under
the existing hourly cap, and nobody needs to press Retry
## Linked Issues or Issue Description
**What happened?**
A task was continued automatically after an interruption. The
continuation run failed during setup, before any provider turn (for
example `setup_failed` with a sandbox HTTP timeout). The stranded
reconciler then set the task to `blocked` with a board-owned recovery
hold. The assigned-work sweep left the task to that hold, and the
fresh-continuation fallback never owns that hold kind. The task stayed
blocked and showed a Retry button. Settled holds whose run was a
recovery run that failed with `adapter_failed`, `setup_failed` or a
provider pack error stayed in the same state, because the held backlog
pass accepted only restart-class codes.
**Expected behavior**
A run that ends for a reason a person does not own is continued again
from saved work. The continuation goes through the fresh-continuation
fallback, so the old workspace epoch is fenced or a new isolated
workspace is used. The per-task limit of 6 automatic continuations per
hour still applies. At the limit the task waits for the next window and
is not blocked.
**Steps to reproduce**
1. Record a fresh continuation for an interrupted run of an
agent-assigned task.
2. Let the continuation run fail with `setup_failed` before provider
work.
3. Run the stranded reconciler. The task becomes `blocked` with a board
hold.
4. Run the assigned-work sweep. It returns `left_to_hold_owner`. No
continuation follows.
**Paperclip version or commit**
master at
|
||
|
|
b5adf35119 |
test(runner): guard that a relaunch after a lost launch acknowledgement leaves one Runner (#21477)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent turns run in a remote Runner (`paperclip-runnerd`) that the server launches in a sandbox. > - A deploy can stop the server after the launch command reached the sandbox but before the server read back the Runner's process identity (a lost acknowledgement). > - The database then has no PID for the run, so the successor server launches the Runner again on the same state folder. > - Two Runners must never serve the same run's state. #20852 proposed a large launch-intent design for this window. Since then, #21079 added a fence that stops a leftover Runner on the same state folder before each launch. > - This pull request adds a Linux regression test that drives the real launcher through the lost-acknowledgement window. > - The benefit is that a future change to the launch path cannot bring back a second Runner without failing this test. ## Linked Issues or Issue Description Refs #20852 (superseded: current master already prevents the duplicate Runner through #21079). Refs #21079. **What happened?** #20852 claimed that a relaunch after a lost launch acknowledgement could start a second Runner for the same run. Nothing proved whether current master still has this problem. **Expected behavior** After a lost acknowledgement, the successor's relaunch stops the orphaned Runner. Exactly one Runner serves the run's state folder, and the process marker names it. **Steps to reproduce** 1. Launch a remote Runner, and lose the launch RPC's acknowledgement before the identity read (the old server stops). 2. Relaunch the same run on the same sandbox from a new launcher (the successor). 3. Count the live `paperclip-runnerd` processes that serve the run's `--state-dir`. ## What Changed - New `server/src/services/native-runtime/remote-runner-lost-launch-ack.test.ts`. It is a test only and changes no product code. - It runs only where `/proc` exists (Linux CI), like the existing #21079 fence tests. - It drives the exported `createRemoteRunnerProcessLauncher` with a local command runner. The old launcher's runner runs the launch command, then throws `remote_launch_ack_lost`. After that, every command fails, because the old server is gone. - A small bash stand-in for `paperclip-runnerd` keeps the Runner's argv[0] and `--state-dir`. Its `reap-lineage` reports a verified, empty lineage. - The test checks these results: the orphan was alive and had no PID on the handle; the successor's Runner has a new PID; the orphan stops; exactly one Runner serves the state folder; the marker names the successor's PID. ## Verification - Linux reproduction (Docker `node:24-bookworm-slim`). I used the exact outer launch command and `REMOTE_RUNNER_CHILD_LAUNCH_SCRIPT`, which I extracted from this head: - master scripts: `runners after successor relaunch [61], orphan alive false`. Result: one Runner (PASS). - the same scripts without the #21079 stale-state fence (counterfactual): `runners after successor relaunch [24, 56], orphan alive true`. Result: duplicate Runner (FAIL). - macOS: `npx vitest run src/services/native-runtime/remote-runner-lost-launch-ack.test.ts` reports the file as skipped (1 skipped), as intended, because there is no `/proc`. - The test runs for real only on Linux CI. CI on this head is the first real run of the vitest file. ## Risks - Low risk. The change is a test only. - The test starts real local processes. Its teardown kills only the PIDs it found on its own temporary state folder, and it never signals PID 0. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use (shell, code search, Docker, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4e2c149f2f |
test(decisions): let the concurrent admission test survive a legitimate serialization retry (#21416)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Decision models let features ask a company AI connection short questions. Each decision reserves budget before it calls the provider. > - The server test "admits concurrent work from the same budget observation" fails or hangs intermittently in CI. This blocks Core image publication from master. > - The test assumed that both admission snapshots always survive. They do not. A legitimate serialization retry can make one admission see the other's committed hold. > - This pull request changes the test so that it proves what the product guarantees, with the same strength, and without the timing assumption. > - The benefit is a deterministic test and unblocked image publication. ## Linked Issues or Issue Description Refs #21387 **What happened?** The test "company decision service > admits concurrent work from the same budget observation and records both obligations" in `server/src/__tests__/decision-models.test.ts` hangs intermittently. CI run 43451649374 (attempt 2, job "verify / General tests (server (4/10))") failed with `Test timed out in 15000ms` at line 371. Before #21387 the same race failed with `expected "vi.fn()" to be called 2 times, but got 1 times`. **Root cause** - `decide()` resolves the credential before admission. `secrets.resolveSecretValue` writes `company_secrets.last_resolved_at` and `updated_at` in its own statement (`server/src/services/secrets.ts:1417-1421`). - Admission runs in a REPEATABLE READ `withOperationAccountingTransaction` (`server/src/services/accounting-transaction.ts:31-53`). Its snapshot starts before it waits on the test's lock. - When decision B's `last_resolved_at` write commits after decision A's snapshot, A's `SELECT ... FOR SHARE` on the same secret row (`server/src/services/decision-models.ts:263-264`) raises 40001. - The retry loop takes a fresh snapshot. If B has committed its 1c hold by then, A sees 1c held against a 1c hard stop and correctly returns `budget_blocked`. A never reaches the provider, so the test waits forever. **Expected behavior** The test passes every time. It still fails if admission becomes a company-wide writer mutex or if the hard stop stops blocking. ## What Changed - Set the test budget to 2c with 1c reservations. Both decisions are now admitted whether or not one admission retries. - Kept the proof that both admissions are in flight at once (both wait on the same row lock at the same time). This is the no-mutex guarantee. - Kept the checks that both providers run concurrently, both holds are `held` at 1c, both decisions succeed, and two cost events exist. - Added a check that a third decision returns `budget_blocked` while both holds are held. - Renamed the test to "admits concurrent work without a company writer mutex and records both obligations". - No product code changed. The retry and the `budget_blocked` result after the retry are correct hard-stop behavior. ## Verification - Root cause reproduced with temporary instrumentation (not committed). Starting decision B after decision A was parked gave 40001 on the `company_secrets` FOR SHARE in 4/4 runs. With the retry delay forced to 300 ms, the old test failed with the exact CI message `expected "vi.fn()" to be called 2 times, but got 1 times`, and the log showed `admitted` then `budget_blocked`. - The new test passes in that forced worst case (staggered starts plus a 300 ms retry delay): both admitted, the third `budget_blocked`. - 50 repetitions (vitest `repeats: 49`, temporary) with 16 extra CPU busy loops on an 18-core machine: 50/50 pass. - Red check: changing the company read in `withOperationAccountingTransaction` to `FOR NO KEY UPDATE` (a company writer mutex) makes the test fail with `expected 1 to be 2`. - `npx vitest run src/__tests__/decision-models.test.ts`: 37/37 in 2 of 3 runs. One run had a failure in a different test ("returns answers and a committed charge before slow budget stop delivery"). That test passes alone and is not changed here. ## Risks - Low risk. Only one test changes. - The test no longer proves that two admissions from the same snapshot both pass a 1c hard stop. That overshoot is not a product guarantee: any concurrent committed write to a share-locked row restarts the snapshot. - Each concurrent decision on the same credential can force a sibling admission to retry because of the `last_resolved_at` write. This is extra latency, not a correctness problem. A follow-up can remove that write from the decision hot path. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use (shell, file edit, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c1ea9304f7 |
fix(recovery): continue Runner preparations whose remote workspace was handed on unchanged (#21481)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Run recovery must continue a task automatically after a deploy, and
a shown Retry must start a run
> - A Runner (paperclip_runner) run can be stopped while it still
prepares its remote sandbox workspace, by a deploy or by a setup failure
> - When the run did not change the workspace, workspace settlement
hands the workspace on at its saved version and turns the run's
workspace reference to `saved`
> - The "no provider work" proof still asks for a workspace-repair
receipt for any run with a remote workspace reference. Only a copy-back
or an abandoned prepared epoch made that receipt. A run released
unchanged never got one
> - So the reconciliation hold never settled, every automatic
continuation was refused, and a person's Retry waited on
`previous_execution_unverified`
> - This pull request accepts the host's released-unchanged receipt for
the exact run, epoch and snapshot as a verified workspace repair
> - The benefit is that these preparations continue by themselves, and
Retry on them starts immediately
## Linked Issues or Issue Description
**What happened?**
On a hosted staging instance, a release swap stopped a Runner run during
preparation. No process started and no provider action happened. The
task showed "Automatic recovery stopped". The automatic continuation
reported "could not start yet" five times. A person then pressed Retry.
The API accepted the Retry, but no run started, and the task showed no
Retry button. A second task failed in setup ("RPC call
environmentExecute timed out") during preparation and had the same
result after Retry.
**Expected behavior**
A run that stopped before any process or provider action continues
automatically after a deploy. Retry on such a task starts a run
immediately.
**Steps to reproduce**
1. Use a `paperclip_runner` agent with a remote sandbox workspace.
2. Stop a run with a graceful shutdown (or a setup failure) while it is
in the `preparing` stage, after it restored the saved workspace version.
3. Let workspace settlement release the workspace unchanged
(`writerRunId: null`, `dirty: false`, an unchanged-baseline receipt for
the run, run reference state `saved`).
4. See that the ownership pass keeps the
`legacy_execution_requires_reconciliation` hold
(`provider_work_not_disproved`), and that a Retry stays deferred with
`previous_execution_unverified`.
**Paperclip version or commit**
Observed on staging at `b0e99dfcfa`; still present on `master` at
`37393f5dc5`.
**Root cause**
`isInterruptedNativePreparation` and `failedNativePreparation` in
`server/src/services/cancelled-native-startup.ts` require
`hasVerifiedWorkspaceRepair` for any run with
`runnerProfileJson.remoteWorkspace`. `hasVerifiedWorkspaceRepair` in
`server/src/services/workspace-recovery-receipt.ts` did not accept the
released-unchanged receipt that `releaseUnchangedRemoteWorkspaceWriter`
writes.
## What Changed
- Add `isReleasedUnchangedRemoteWorkspace` in
`server/src/services/remote-workspace-abandoned-preparation.ts`. It is
true only when the run's reference is `saved`, and the workspace's
unchanged-baseline receipt names this run, the same epoch and the same
snapshot, with a release time.
- `hasVerifiedWorkspaceRepair` accepts that receipt, next to the
copy-back receipt and the abandoned prepared epoch.
- Add regression tests for the deploy-interrupted shape (held
preparation settles and continues; Retry starts) and for the
setup-failure shape (Retry starts). Add a negative test: a preparation
that still holds its workspace unsaved keeps its hold.
## Verification
- Red without the fix, green with it:
- `cd server && npx vitest run
src/__tests__/interrupted-preparation-hold-settle.test.ts` (32 passed;
the 2 new positive tests fail without the fix)
- `cd server && npx vitest run
src/services/legacy-preparation-retry-hold.test.ts` (6 passed; the new
test fails without the fix)
- Related files pass unchanged:
`src/services/saved-work-retry-eligibility.test.ts` (8),
`src/__tests__/interrupted-preparation-restore-retry.test.ts` (7).
- Focused test files only (the machine was loaded). CI supplies
typecheck, build and the full suite.
## Risks
- Low risk. The new proof applies only when the host itself released the
run as the unchanged writer of that exact epoch and snapshot. A run that
still holds unsaved files, a later writer that replaced the receipt, or
any provider execution evidence still keeps the hold.
- Not changed: the retry API still answers "accepted" for a saved Retry
intent that admission later refuses for another reason. That is a
separate follow-up.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use
(shell, file edits, test execution).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
a48eaaef57 |
perf(server): skip the agent row lock for unchanged agent-file checkpoints and upload before the lock (#21457)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent keeps its own files (AGENT_HOME). A warm native session saves these files at the end of each turn (an agent-file checkpoint) > - Each checkpoint took the agent row lock (`agents ... FOR UPDATE`) on the small file-storage pool, also when the run changed no file > - A checkpoint that changed files also uploaded the new bytes to object storage while it held that lock. Instruction edits and other runs of the same agent waited for the upload > - With many agents, these long transactions use the 2-connection file-storage pool and slow every agent > - This pull request removes the lock from an unchanged checkpoint and moves the uploads of a changed checkpoint before the lock > - The benefit is that an unchanged checkpoint takes no row lock, and the row lock is no longer held across object-storage I/O ## Linked Issues or Issue Description **What existing behavior does this improve?** The warm agent-file checkpoint: `checkpoint()` in `server/src/services/agent-directory-working-copies.ts`, `agentFileStore.locked` and `apply` in `agent-file-store.ts`, and `persistDurableAgentFiles` in `agent-file-durability.ts`. **Current behavior** - When the run changed no file, the checkpoint still ran `store.locked(..., write = true)`. That transaction locks the agent row (`FOR UPDATE` and `FOR NO KEY UPDATE`), restores and invalidates the local projection, walks the canonical tree, and compares the durable head. Its only result is the authorization check. - When the run changed files, `persistDurableAgentFiles` uploaded each new file to object storage inside the transaction that holds the agent row lock. **Proposed behavior** - An unchanged checkpoint (`saveUnchanged`) first reads, without a lock, whether the locked file-tree step would change anything (`agentFileProjectionCurrent`): the durable head exists and the local projection is marked at its generation, or durable files are off, no head exists and the local adoption is recorded. If so, it runs only the write authorization (`authorizeSave`): the agent interaction policy and `authorizeInstructionCommit`, in a short transaction without row locks that always rolls back, so it writes nothing. If not, it runs the locked save as on `origin/master`, which adopts, restores or heals the durable head. A de-authorized agent still gets the same error (the existing test "rechecks authorization even for an unchanged warm checkpoint" still passes). - A changed checkpoint runs the same read-only authorization (it consumes no accepted change confirmation), then uploads the run's changed files before it takes the lock (`uploadAgentFilesBeforeCommit`). A run that lost its save permission uploads nothing. The locked commit runs the authorization again, and only that commit consumes a confirmation. The locked commit uses an uploaded object only for a file with the same SHA-256 and size. It uploads any other file itself, as before. **Reason and benefit** The target is 1,000 agents in one controller process. Most warm turns change no agent file. These turns now take no row lock and send fewer statements. A changed turn holds the row lock for less time, because object-storage latency is outside the lock. ## What Changed - `agent-file-store.ts`: add `authorizeSave(companyId, agentId, actor)` and `saveUnchanged(companyId, agentId, actor)`. `apply` calls it for a checkpoint, then uploads the run's changed files before `locked`, and passes the uploaded objects to `persistDurableAgentFiles`. `locked` takes an optional map of uploaded objects. - `agent-file-durability.ts`: move the verified upload stream into `uploadAgentFile`. Add `uploadAgentFilesBeforeCommit`. It reads the durable head without a lock, skips files the head already has at the same path and digest, and uploads the others. If an upload fails, it stops and the locked commit uploads the rest, so errors are the same as before. `persistDurableAgentFiles` uses an uploaded object when the canonical file has the same digest and size. - `agent-directory-working-copies.ts`: an unchanged checkpoint calls `saveUnchanged` instead of `store.locked(..., true)`. - `agent-file-durability.ts`: add `agentFileProjectionCurrent` (one head read and two file checks, no lock). - The commit is still inside the agent row lock. The head cannot change between the read and the write of the commit, so it needs no separate compare-and-set. The upload before the lock does not need one either: an object is immutable and verified by digest, and the commit records it only for a file with that digest. ## Verification - New `server/src/__tests__/agent-file-checkpoint-locking.test.ts` (embedded PostgreSQL, object-storage fixture). All tests fail on the code before the change and pass with it: - An unchanged checkpoint completes while another transaction holds the agent row lock. It sends no `FOR UPDATE`, `FOR NO KEY UPDATE` or `FOR SHARE` statement and only one unlocked read of `agent_file_roots`. - Durable files on, the agent has local files and no head: an unchanged checkpoint creates the head, as on `origin/master` (added after review; failed without the fallback). - The local projection is not marked at the head: an unchanged checkpoint takes the locked path and marks it again (added after review; failed without the fallback). - A changed checkpoint uploads its file while the agent row is not locked (the fixture checks the lock with `FOR UPDATE NOWAIT` from another connection during each upload). The head then records that object. - A run loses its save permission before a changed checkpoint. The checkpoint is refused and uploads nothing (added after review; I saw it fail without the authorization before the upload). - An agent with `agents:suggest-changes` and one accepted change confirmation saves a changed checkpoint. An unchanged checkpoint first leaves the confirmation unconsumed. The changed save succeeds once and consumes the confirmation once; the next change is refused before it uploads (added after review; I saw it fail when the preview committed). - Two warm runs of the same agent save at the same time. Both files are in the canonical tree and in the head, and every head object has the correct digest. - An instruction edit commits while a checkpoint waits in its upload, without a wait for the checkpoint. Then the checkpoint commits. Both changes are in the head. - A save stops between upload and commit (the instruction entry changes while the bytes upload, so the commit refuses). The head does not change and stays restorable. The next save commits the same file. The unused object is never authoritative; the existing object collector removes it after its grace period. - Existing `agent-directory-working-copies.test.ts`, `agent-file-durability.test.ts` and `agent-file-checkpoints.test.ts` pass. - Measurement (opt-in, `PAPERCLIP_AGENT_FILE_CHECKPOINT_BENCHMARK=1`, test "measures checkpoint statements and agent row lock time"): one warm copy with 51 files; 10 unchanged checkpoints; then 5 checkpoints that each change one file, with 250 ms object-store latency. Wall times are noisy (load average about 40). Statement counts are exact. | | Before (`origin/master`) | After (final head) | | --- | --- | --- | | Unchanged checkpoint: statements | 26 | 25 (one unlocked head read, and the read-only authorization) | | Unchanged checkpoint: row-lock statements | 2 | 0 | | Unchanged checkpoint: median time | 214 ms | 26–126 ms | | Changed checkpoint: statements | 29 | 47 (the read-only authorization before the upload and one head read) | | Changed checkpoint: median time the agent row is locked | 924 ms | 51–227 ms | | Changed checkpoint: uploads while the agent row is locked | 1 | 0 | The "after" ranges come from two runs at different machine load (load average between about 18 and 40). ## Risks - Master side effects of the locked unchanged save, and what this change does with each: - Kept: the write authorization (`assertAgentInteraction`, `authorizeInstructionCommit`), so a run that lost its permission still gets the same error. - Kept through the fallback: one-time adoption (no head yet), restore of a projection that is not marked at the head, re-mark after a failed persist (a write path removes the marker before it changes files), and heal of a head that lags the root after such a failure. `agentFileProjectionCurrent` returns false in each of these cases, so the locked save runs. - Dropped when the projection is current: the re-mark at the same generation (no change), and the compare of the local tree with the head. With the marker at the head generation, every writer that changed the tree went through the locked save and moved the marker, so the compare finds no difference. An edit made outside Paperclip directly in the canonical directory is not picked up by an unchanged checkpoint any more; the next save that goes through the locked path still records it. - Dropped when the projection is current: the storage-limit check of the canonical tree in `persistDurableAgentFiles`. An over-quota agent whose run changed nothing no longer gets an error from that check; any save that changes files still enforces the limits. - Dropped: the `FOR NO KEY UPDATE` locks that `assertAgentInteraction` takes on the actor and target agents (`lock: "update"`). Those locks hold permission changes off a write until it commits. The read-only authorization writes nothing, so there is nothing to hold them off. The locked commit of a changed save still takes them. An unchanged save has no mention texts, so master locked no other agents for it. - Effects of `authorizeSave` outside the rolled-back transaction: none. I read `assertAgentInteraction` (`agentInteractionDenial`: reads only), `authorizeInstructionCommit`, `resolveInstructionActor` (reads only), `authorizationService.decide` (reads only; its memo maps live on the bound actor and on the service instance, both created for this call and dropped after it), and `changeConsentGateService.assertConsented` (one `UPDATE` of the confirmation row, rolled back). None of them publishes live events, queues notifications or wakes, writes activity through another connection, makes external calls, or changes an in-process cache that outlives the call. - Behavior change: on `origin/master`, an unchanged checkpoint of a consent-gated agent (`agents:suggest-changes`) consumed an accepted change confirmation, because the locked transaction committed. Now the fast path does not; only the commit of a changed save consumes it. When the fast path cannot be used (adoption, restore or heal), the locked save runs as on master and can still consume a confirmation. - Not in this change: the manifest is still stored twice in the receipt (`baseline` and `checkpointManifest`). Servers of the previous version read `receipt.baseline` during a deploy overlap, and the next capture needs the stamps in `checkpointManifest`, so neither can be removed safely now. - Not in this change: on the volume-free server, `serial()` still opens a dedicated connection with an advisory-lock transaction for each checkpoint. That lock serializes controllers and is not a row lock; changing it is a separate design decision. - A changed checkpoint can now leave an unused object in storage when its commit is refused after the upload (for example, the instruction entry changes during the upload) or the process stops before the commit. This was already possible for a failed commit. Such an object is never in the durable head, so no reader can reach it, and the existing `collectAgentFileObjects` maintenance removes it after `AGENT_FILE_OBJECT_GRACE_MS`. A permission that was removed before the save stops the save before any upload. - Old and new servers can run at the same time: the head format and the receipt format do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, file edits) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8f105e743a |
fix(accounting): recover closed legacy runs without inventing usage (#21493)
Preserve exact physical process closure independently from incomplete cost reporting. Recover a missing receipt through the existing authorized accounting Retry without replaying provider work. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
11f4a76743 |
perf(native-runtime): share one frozen assigned tool catalog across sessions (#21482)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One server process controls every native agent session. Each session gets the app tools (MCP tools) that are assigned to its agent. > - Each live or warm session kept two private copies of its assigned tool catalog: the gateway descriptors and a canonical copy of every input schema. > - With 400 tools of about 5.7 KB each, that was about 7.6 MB of heap for each session. With 800 tools, it was about 15 MB. Warm sessions stay for 5 minutes after the last turn, so idle agents also hold this memory. > - At 1,000 concurrent agents, this is 7 to 15 GB of heap, and the server cannot get there. > - This pull request keeps one deeply frozen catalog for all sessions of the same company that get the same admitted tools and schemas. > - The benefit is that each additional session on the same catalog adds about 40 to 80 KB of heap instead of 7.6 to 15 MB, and session start is faster. ## Linked Issues or Issue Description Refs #21415 (releases a finished session's tool authority). Refs #21440 (adds the catalog memory benchmark that this change extends). **What existing behavior does this improve?** `createAssignedMcpTools` in `server/src/services/native-runtime/assigned-mcp-tools.ts`. Each call kept every gateway descriptor (with its provider metadata) and a separate canonical clone of each input schema. Each `definitions()` call also made a new `structuredClone` of every schema. **Subsystem affected** Native runtime: assigned app tools (MCP) for native agent sessions. **Current behavior** Retained heap grows by one full catalog for each session. Measured with the real `createAssignedMcpTools` (5.7 KB schemas): 400 tools is 7.58 MB per session, and 800 tools is 15.1 MB per session, for 1, 10, and 100 sessions. Each `definitions()` call takes 14 to 41 ms because it clones every schema. **Proposed behavior** Sessions of the same company that get identical admitted tools share one frozen catalog. The total for 100 sessions is about one catalog: 4.2 MB at 400 tools and 8.1 MB at 800 tools. `definitions()` takes 0.01 ms. **Reason and benefit** The server must hold 1,000 concurrent agents in one process. The catalog was the largest per-session heap cost. Revocation and authorization do not change: every call is still authorized again by the gateway, and every new session still reads a fresh listing. **Breaking changes** None in the API or data. The direct definitions that `definitions()` returns are now frozen. A consumer that writes to a returned definition or schema gets a `TypeError` and does not change other sessions. No production consumer writes to them (see Risks). ## What Changed - `server/src/services/native-runtime/assigned-mcp-tools.ts`: - A session keeps only a small frozen record for each tool (gateway name, display name, description, risk), plus the frozen schema and the frozen definition. It does not keep the gateway descriptors. - New per-process index of shared catalogs. The key is the SHA-256 of an identity string: `[companyId, [[assignedName, gatewayName, displayName, description, risk, sha256(JSON of the input schema)] for each listed tool, in listing order]]`. On a hit, the code also compares the full identity string, so a hash collision cannot give a wrong catalog. - The identity is computed from the fresh listing that the gateway returns for this session. The gateway applies all per-agent, per-run, per-profile, and policy filtering, and the approval description suffix, before it returns the listing. Thus two sessions share only if they were given exactly the same tools, names, descriptions, risks, and schemas in the same company. Without `companyId`, nothing is shared. - Schemas, definitions, tool records, and the definition arrays are deeply frozen. Search results still return cloned copies. - Sessions that start at the same time with the same catalog wait for one build and share it. - The index holds each catalog through a `WeakRef`, so it never keeps a catalog alive. The index has at most 512 entries and 256 MiB (accounted schema and description bytes). The oldest entries are removed first. A catalog that is larger than the byte limit is not indexed. - New `close()` on the assigned tools. When the last session that holds a catalog closes, the index entry is removed. A `FinalizationRegistry` also removes entries whose catalog was collected. - `definitions()` returns the shared frozen definitions for the session's work mode. It no longer clones every schema on each call. - `server/src/services/native-runtime/paperclip-runner-tool-authority.ts`: `close()` also closes the assigned tools. The session executor already closes the authority when it revokes a session's tool authority. - `server/src/services/native-runtime/native-session-executor.ts`: one line. It passes the run binding's `companyId` to `createAssignedMcpTools`. - `server/scripts/benchmark-assigned-mcp-tools-memory.ts`: passes `companyId`, adds `--companies`, measures session start and `definitions()` time, and returns fresh copies of fixed rows (like database reads). - Tests in `assigned-mcp-tools.test.ts` and `paperclip-runner-tool-authority.test.ts`. ## Verification Red/green tests in `server/src/services/native-runtime/assigned-mcp-tools.test.ts` ("shared assigned tool catalogs"): - Retained heap: 20 sessions of one company on a 60-tool, 4 KB-schema catalog. The heap for the other 19 sessions must be less than two catalogs. Red on master: `expected 14025976 to be less than 3788112`. Green with this change. - Shared deeply frozen schemas, also for sessions that start together. Red on master (`Object.is` equality fails). - Mutation isolation: writes to a returned definition, schema property, and `required` array, and a delete, throw `TypeError`. Changes to the gateway's source rows and to a search result do not change any session. Red on master (no throw). - Release and bounds: the entry stays while one session holds it and goes when the last session closes. `close()` is idempotent. A new session after release builds a new catalog. 517 companies keep the index at 512 entries or fewer. Red on master (no `close` or stats). - Guard tests: two companies with identical catalogs do not share. Sessions without a company do not share. Fewer tools, an edited schema, an edited description, or a different risk do not share. A revoked or edited tool is served from fresh discovery. On master these pass, because master never shares. To prove they test the key, I removed `companyId` and the schema digest from the identity: three guard tests failed, and they pass with the full key. - An existing test changed: "does not project gateway secrets..." now expects a write to a returned schema to throw `TypeError` (before, the write changed only a private clone). The isolation check after the write is the same. - `paperclip-runner-tool-authority.test.ts`: `authority.close()` closes the assigned tools. Heap (post-GC `heapUsed` delta, `node --expose-gc`, 5.7 KB schemas, one company). Command: `cd server && node --expose-gc --import tsx scripts/benchmark-assigned-mcp-tools-memory.ts --tools <N> --schema-bytes 5700 --sessions <S> --turns 5`. | Tools | Sessions | Total before | Per session before | Total after | Per session after | | --- | --- | --- | --- | --- | --- | | 400 | 1 | 7.60 MB | 7.60 MB | 3.96 MB | 3.96 MB | | 400 | 10 | 75.9 MB | 7.59 MB | 4.04 MB | 0.40 MB | | 400 | 100 | 757.9 MB | 7.58 MB | 4.18 MB | 0.042 MB | | 800 | 1 | 15.2 MB | 15.2 MB | 7.91 MB | 7.91 MB | | 800 | 10 | 151.4 MB | 15.1 MB | 7.98 MB | 0.80 MB | | 800 | 100 | 1,513 MB | 15.1 MB | 8.13 MB | 0.081 MB | The first session also uses about half the heap of before, because the session no longer keeps the gateway descriptors. When all sessions are dropped without `close()`, the catalog is collected (0 bytes held after release). Session start, interleaved A/B in one process (old and new `createAssignedMcpTools` alternate, 40 starts each, median of the last 35, includes a fresh copy of the listing; the machine had a load average of 25 to 65): | Tools | Listing copy only | Before | After | | --- | --- | --- | --- | | 400 | 22.6 ms | 68.0 ms | 37.6 ms | | 800 | 30.5 ms | 72.5 ms | 41.2 ms | A start that reuses a catalog does no schema clone and no admission. It still serializes and hashes each schema to compute the key. The first start of a new catalog does the same admission work as before (the cold first start was about 600 to 750 ms in both versions on this busy machine). `definitions()` median: 14 to 41 ms before, 0.01 ms after. Other checks: - `cd server && npx vitest run src/services/native-runtime/assigned-mcp-tools.test.ts src/services/native-runtime/paperclip-runner-tool-authority.test.ts src/services/native-runtime/runner-api.integration.test.ts src/services/native-runtime/runner-tool-results-prp.integration.test.ts src/__tests__/deployment-connections.test.ts`: 204 passed. - `cd server && npx vitest run src/services/native-runtime/native-session-executor.test.ts -t "assigned|tool authority|released|MCP"`: 31 passed. - `cd server && npx vitest run src/__tests__/chat-channels.integration.test.ts -t "assigned"`: 10 passed. - `cd server && npx tsc --noEmit`: pass. - `pnpm check:module-boundaries`: pass. - `pnpm verify` selects a broad plan (the benchmark script is a broad input), so CI supplies the full suite. ## Risks - Frozen definitions: the direct definitions now go to the runner drivers as frozen objects. I checked the consumers. `PaperclipRunnerToolAuthority.definitions()` filters and spreads them. The Codex, Cursor Cloud, OpenCode, ACPX, and runnerd drivers read them, spread them, clone them (`structuredClone` in ACPX and the OpenCode MCP bridge), or serialize them. Their types are `readonly Readonly<Record<string, unknown>>[]`. The first-party semantic tool definitions are already shared module constants that go through the same path without clones. If a consumer that I did not find writes to a definition, it gets a `TypeError` instead of a silent change. Catalogs over 256 tools or over the listing budget use the two on-demand tools and are not affected. - Authorization: the shared data is only the result of a listing that the gateway already authorized for that session, keyed on the full content plus the company. Execution still goes through the gateway with the session's own token on every call. Search still intersects with a fresh listing. A new session still reads a fresh listing, so a revoked or edited tool takes effect for new sessions and for existing sessions at the same points as before. - Several server processes: the index is a per-process memo of the listing content. Each process builds its own catalog from its own listing. Nothing is stored or shared outside the process. - The index is bounded (512 entries, 256 MiB) and holds catalogs only weakly. Index eviction only stops future sharing. It never frees memory that a session uses. - No schema change. No migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use (shell, file edit, test runs, heap and timing benchmarks). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (no documented behavior or command changes) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
533d02ff1a |
perf(recovery): match scoped deadline entries only against scopes of their own company (#21467)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Live-run recovery uses the shared scoped deadline worker (`scoped-deadline-work.ts`). Each queue keeps its entries (one for each running run) and the scopes that writes marked dirty (for example one company and one run) > - To schedule and to pick due work, the worker tested every entry against every dirty scope. It also decoded each scope hint with `JSON.parse` again inside those loops, and each reconstructed scope walked all entries to drop vanished ones > - With many runs and many dirty run scopes, this work grows with entries × scopes on the event loop of the one server process that controls all runs > - This pull request lets an owner declare a partition key for its scopes. Live-run recovery uses the company. The worker then tests an entry only against the scopes of its own company, and decodes each hint once > - The benefit is that matching cost grows with entries + scopes, with the same results ## Linked Issues or Issue Description Refs #21444 (the lease renewal change from the same load model). **What existing behavior does this improve?** The scoped deadline worker in `server/src/services/scoped-deadline-work.ts`, which live-run recovery (`server/src/services/heartbeat/live-recovery-work.ts`) uses for its seven queues. **Subsystem affected** Server: background work scheduling for live-run and resource recovery. **Current behavior** For 400 entries in 40 companies and 40 dirty run scopes, one schedule computation and one pass call `matches` 27,830 times and decode each hint many times. **Proposed behavior** The same case calls `matches` fewer than 3,200 times and decodes each hint at most once. Owners that do not declare a key behave exactly as before. **Reason and benefit** The plan for 1,000 concurrent agents needs per-run background work that does not grow with the product of runs and changes. The work runs on the main thread of the controlling process. **Breaking changes** None. `scopeKey` is optional. ## What Changed - `scoped-deadline-work.ts`: new optional `scopeKey` option (`entry(entry)` and `scope(scope)`). The contract: a scope with a defined key matches only entries with the same key. A scope without a key (for example `*`) is tested against every entry, as before. - The worker indexes the dirty scopes by key, so `next()` and the due filter test each entry only against the scopes of its key and the unkeyed scopes. - When a reconstructed keyed scope drops vanished entries, it visits only the entries of its key (an index built once for each pass). - Scope hints are decoded once and the decoded value is reused while the hint is dirty or scanning. - `live-recovery-work.ts`: declares the company as the key. Its `matches` already requires the same company for every defined scope. ## Verification - New test in `scoped-deadline-work.test.ts`: 400 entries in 40 companies, then 40 run-scoped invalidations. It requires fewer than 3,200 `matches` calls and at most 40 decodes, and the next deadline is unchanged. - Red first: with `origin/master` `scoped-deadline-work.ts`, the test failed with 27,830 `matches` calls. - `pnpm exec vitest run src/__tests__/scoped-deadline-work.test.ts`: 22 passed. - The live-run recovery matrix (`live-recovery-work.test.ts`) is covered by CI. ## Risks - If an owner declares a key that its `matches` does not respect, the worker can miss a match. Only live-run recovery declares a key, and its `matches` returns false for any defined scope of another company. - Low risk otherwise: owners without a key keep the old code path. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
754b1f36d3 |
perf(native): renew all of a process's execution leases in one transaction, without a recovery wake (#21444)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One server process controls every running native agent. Each run holds a native execution lease that the process must renew before it expires (60 seconds for a remote run, 20 minutes for a local run) > - Each run renewed its lease on its own timer, in its own transaction, on a private pool of two connections. At 1,000 remote runs that is about 50 transactions each second through two connections > - When renewals queue behind each other, a renewal can miss its 60 second deadline, and the run is lost although the server is healthy > - Each renewal also woke six live-run recovery queues for its run. The recovery owner then re-checked a healthy run about every 20 seconds, although a renewal only moves the deadline later > - This pull request renews all leases of one process with one shared timer. All leases that are due go into one transaction of fixed size, the same model that the legacy controller lease already uses. A renewal no longer wakes recovery, as #20924 did for the legacy lease > - The benefit is that the renewal cost no longer grows with the number of runs, and all safety properties of a single renewal stay the same ## Linked Issues or Issue Description Refs #20924 (the same change for the legacy controller lease), #21242. **What existing behavior does this improve?** Native execution lease renewal in `server/src/services/native-runtime/native-execution-lease.ts`. Each run had one `setInterval` and one transaction per renewal: statement timeout, row lock, recovery notification, update, commit. **Subsystem affected** Server: native runtime, controller lease renewal on the liveness connection pool. **Current behavior** The renewal cost grows with the number of runs. A local load model with 1,000 remote runs sends 35.7 lease transactions and 212.8 lease statements each second, and the event-loop utilization is 0.34. All of them use a pool of two connections. **Proposed behavior** One timer for each database client renews all due leases in one transaction for each 200 runs. With 1,000 remote runs the same model sends 0.29 lease transactions and 1.7 lease statements each second, and the event-loop utilization is 0.07. **Reason and benefit** The plan for 1,000 concurrent agents needs renewals that do not scale with the run count (item 1d). A renewal that misses its deadline loses the run. **Breaking changes** None. The `NativeExecutionLeaseRenewal` contract (`stop()`, `remainingMs()`) and `renewNativeSessionExecutionLease` do not change. ## What Changed - `native-execution-lease.ts`: a renewal (batched or single) no longer calls `notifyLiveRunRecovery`. A renewal only moves the lease deadline later. Recovery already holds a deadline at the earlier expiry. At that deadline it re-reads the row and moves to the new expiry. A lapsed lease is still found at its expiry. This change does not touch the signals of the writers that release, revoke or take over a lease, or that change the run status. This also removes six notifications for each renewal, which was 1,200 notifications for a batch of 200 runs during a deploy overlap. - `native-execution-lease.ts`: one shared schedule for each database client replaces the timer of each run. When the first lease is due, all leases that are due within a quarter of their own interval renew with it. Thus the leases of a process converge on one batch for each interval. - A batch is one transaction on the liveness pool for each 200 runs: `statement_timeout`, one `SELECT … FOR UPDATE SKIP LOCKED` of the coordinator rows, and one `UPDATE … FROM (VALUES …)`. - The batch keeps the guarantees of a single renewal: - It locks each coordinator row before it reads the expiry, as before. Thus an owner whose lease expired while another writer held the row is never revived. - It extends a row only for the exact owner: run, company, task, lease owner, attempt, controller boot id, pid, process start, and a live lease. - The confirmed deadline is sampled after the locks and before the update, as before. - A run whose row was locked and not extended fails with `native_session_lease_lost`. The other runs in the batch succeed. - These rows do not renew in the batch. They renew alone through the unchanged single-run path: - A row that another writer holds (`SKIP LOCKED`). Thus one busy row cannot hold the batch until the statement timeout. - A row that the lock did not find (another owner, or no row). The single path gives the same answer as before. - A local lease that lapsed (host sleep). Only the single path may take it back, under the run row lock. - A batch that is cancelled (statement timeout `57014` or deadlock `40P01`) extends nothing. Its runs retry after 1 second, and each run keeps its own deadline timer. Before this change, a statement timeout waited for the next 20 second tick. - Any other batch failure, also an error thrown while the statement is built, may come from one bad row. Each run of that batch then renews alone at once through the single path, so a bad row can cost only its own run. A run that fails alone while others in its batch renew keeps renewing alone (out of later batches) until a single renewal succeeds. During a database outage the single renewals also fail; those runs return to the batch at their next interval, not after a 1 second retry, so an outage causes at most one queued single transaction for each run and interval. - Runs that become due while a batch is in flight wait for that batch. A batch that runs for more than 15 seconds no longer holds back the next batch. - A tick that passed while a renewal was in flight is skipped, as a fixed interval skips it. Thus a slow database gets no extra load. - The shared timer replaces the interval timer of each run. It exists only while this process holds a lease, and it fires at the next renewal deadline. With no held lease there is no timer and no SQL. - New export `renewNativeExecutionLeasesNow(db)` renews all leases of the process at once. Tests use it. ## Verification - New PostgreSQL tests in `native-execution-lease-batch.test.ts`: - 3 runs and 60 runs each renew in 1 transaction with the same number of statements. - 450 runs renew in 3 transactions. - In one batch, the runs whose lease was taken over, whose controller boot changed, or whose lease expired fail with `native_session_lease_lost`. Their rows do not change. The healthy runs renew. A newer attempt under the same owner is not lost, as before. - Interleaving: another transaction holds a coordinator row from before its lease expires until after it expired. The healthy run in the same batch renews at once. After the lock is released, the expired owner is lost and its row stays expired. - A takeover that commits before the batch: the batch extends nothing and the run is lost. - A local and a remote lease renew in one transaction, each for its own lease length. - A poison binding (an attempt outside the integer range) makes the shared UPDATE fail with `22003`. The other 4 runs still renew in the same tick, only the poison run is lost, and the next tick renews the 4 runs in 1 transaction again. - An invalid date throws while the statement is built. The other runs renew, the broken run is lost alone, and no lease stays busy (the next call renews again in 1 transaction). - New PostgreSQL test in `live-recovery-work.test.ts` with the real registered recovery owners: another server's native run renews its lease. No recovery pass runs after the renewal. At the old expiry recovery re-reads the row, keeps the run running and moves its deadline to the new expiry. When renewals stop, recovery acts on the run within 10 seconds after the end of the last lease. - Red first: with a shim on `origin/master` that renews each lease alone, the transaction test failed (`expected 3 to be 1`) and the 450-run test failed. - The unit tests in `native-execution-lease.test.ts` now drive the batch. New tests show that a non-cancelled batch failure renews the run alone at once, and that an outage (batch and single renewal both fail) does not retry every second and loses no run. - `native-execution-lease-reacquire.test.ts`: the host-sleep tests now go through the batch, which hands the lapsed local lease to the single path. - Load model (`startNativeSessionExecutionLeaseRenewal` for N remote runs that start at random times over 20 seconds, 70 second window, local PostgreSQL, machine load average about 50): | Runs | Lease transactions/s before → after | Lease statements/s before → after | Event-loop utilization before → after | Smallest margin to the 60 s deadline | Lost | | --- | --- | --- | --- | --- | --- | | 50 | 1.81 → 0.11 | 10.9 → 0.69 | 0.015 → 0.095 (noise) | 40.0 s → 40.0 s | 0 → 0 | | 1,000 | 35.67 → 0.29 | 212.8 → 1.71 | 0.341 → 0.065 | 39.0 s → 40.0 s | 0 → 0 | The window includes the 20 second ramp, so the steady rate before is higher (one transaction each 20 seconds for each run, 50/s at 1,000 runs). After the change, one batch carries up to 200 runs. ## Risks - **Behavior change: batch-wide error handling.** One shared transaction now carries up to 200 runs, so one database error reaches all of them. A cancelled batch only skips the extension and retries after 1 second. Any other batch error sends each run to the single path at once, so one bad row (a bad value, a cast error, an error while the statement is built) costs only its own run. In a database outage, up to one single transaction for each run waits on the two-connection pool at each interval. Each run's own deadline is unchanged. - **Behavior change: no recovery signal on renewal.** See below. - During a deploy overlap the old server renews its runs one by one and the new server renews in batches. Both use the same row lock and the same conditions, so they do not conflict. Rows written by the old version have the same columns. - `SKIP LOCKED` sends a busy row to the single path, which waits for the lock as before. - **No recovery signal on renewal:** recovery of a healthy run of another server now runs once for each lease period at the old expiry, not after each renewal. A dead run is found at the same time as before: at the expiry of its last lease. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
969cf246ae |
perf(run-events): keep each native source's cursor on the run row (#21435)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One server process stores every event of every running native agent in `heartbeat_run_events` > - Each native source event append counted all retained events of its source (`count(*)`, `max(source_seq)`) to compute the acknowledgement cursor > - So the cost of each append grew with the length of the run, and long runs made every text delta slower > - This pull request keeps the gap-free cursor of each source on the run row and moves it in the same UPDATE that reserves the event sequence > - The benefit is that an in-order append stops reading other events, so its cost does not grow with run length ## Linked Issues or Issue Description Refs #21131 (earlier reduction of statements per event). Item 1b of the agent scaling plan in #21418. **What existing behavior does this improve?** The append of a native source event (`appendNativeSourceEvent` in `server/src/services/heartbeat-run-events.ts`). It ran a `retained` CTE that counted every event of the source on every append. **Subsystem affected** Server run event ingest (`heartbeat_run_events`, `heartbeat_runs`). **Current behavior** On a run that already holds 50,000 events, one in-order append takes about 10–13 ms (p50) on a local PostgreSQL. On a run with 400 events it takes about 3.3 ms. **Proposed behavior** The append reads the stored cursor in the run lock statement that it already sends. An in-order event moves the cursor and does not count events. On the same 50,000-event run an append takes about 1.7–3.3 ms (p50). **Reason and benefit** Text deltas are about 85% of the server's statements in the load model. Their cost must not grow with run length. **Breaking changes** None. The new column is nullable. The previous release ignores it. ## What Changed - Add the nullable column `heartbeat_runs.native_source_cursors` (`jsonb`, `{"<source instance id>": <seq>}`). Migration `0479` is `ADD COLUMN` only and classifies as `online_compatible`. - The run lock read of a native append also reads the cursor of its source. - When the event sequence is at most cursor + 1, the append statement has no `retained` CTE. It moves the cursor in the UPDATE that reserves the sequence, and the receipt uses the cursor. - Every other case (no cursor, a gap, or a cursor behind the stored events) runs the earlier recount. In the same UPDATE it stores the cursor when the source has no gap, and removes it when the source has a gap. So the event that fills a gap recounts. - The cursor is a lower bound. A writer that does not keep it (the previous release during a deploy overlap, or `NativeRunCoordinatorStore`) can only make it lag. The next out-of-order event then recounts and corrects it. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-run-event-atomic-append.test.ts src/__tests__/heartbeat-run-event-sequencing.test.ts` passes (19 tests). - Red/green: before the change, 3 tests failed: the two new tests and the updated lock-read statement test. After the change, all pass. - New: "keeps the source cursor on the run so an in-order append does not count the run's events" checks that the append statement has no `count(` and that a replay returns the cursor. - New: "recounts a source whose run has no cursor or a cursor behind its stored events" covers rows saved before the change, rows from a writer that does not keep the cursor, a gap, and the event that fills the gap. - Focused related tests: `paperclip-control-plane-port.test.ts`, `native-observer-replay`, `native-event-boundary-golden`, `chat-native-progress-work`, `live-recovery-work`, `native-interaction-bridge`. - `pnpm check:module-boundaries` passes. `pnpm verify` returned exit 2 (broader CI coverage required) because of local benchmark scripts that are not in this pull request. Required CI supplies the broad coverage. - Append cost on a run with many events (local embedded PostgreSQL, 1,000 in-order appends, busy machine, load average about 20–40): | Events already in run | Before p50 / p90 | After p50 / p90 | | --- | --- | --- | | 50,000 (run 1) | 10.4 / 15.8 ms | 1.7 / 3.7 ms | | 50,000 (run 2) | 12.9 / 21.6 ms | 3.3 / 5.9 ms | | 400 | 3.3 / 6.2 ms | 3.7 / 7.7 ms (noise) | - Statements per event do not change: BEGIN, run lock read, one append statement, COMMIT (4). The append statement no longer scans the source's events. - Load model (`scripts/benchmark-agent-scaling-load.ts` from #21418, `AGENTS=50 SECONDS=20`): the machine was too busy for a stable result. Deltas only: before 20,000 of 20,000 done at 3,999 statements/s. After: 13,796 and 14,817 of 20,000 in two runs. All sources: before 6,346 of 20,000 deltas done. After: 20,000 and 2,083 in two runs. A 20-second window gives only 400 events per run, so it cannot show this change. The long-run figures above are the evidence. ## Risks - Low. The column is nullable and the previous release does not read it. Runs that are in flight during a deploy have no cursor. Their first append recounts once and stores the cursor. - A stale cursor can only make the acknowledgement lag, never pass the stored events. Event rows are deleted only with their company or agent. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0d832a02d3 |
perf(ci): shorten the general server and chat test lanes (#21393)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the `PR` CI workflow before it can merge > - The `General tests (server (N/12))` and `General tests (chat (N/3))` jobs finish last, so they set the wall-clock time of CI > - Before this change, server shards took 9.5 to 11.7 minutes and chat shards took 6.4 to 9.6 minutes; the target is 7 minutes or less > - Measurement shows that the chat lane was unbalanced and that the server lane is limited by CPU on the 4-vCPU runners > - This pull request balances the chat lane by recorded test time, runs three chat partitions per job, and takes general server weights from PR runs > - The benefit is a chat lane under 7 minutes and measured evidence for the shard count that the server lane needs ## Linked Issues or Issue Description **What existing behavior does this improve?** The PR CI general server and chat test jobs. They are the slowest jobs of every pull request. **Subsystem affected** CI test scheduling: `scripts/run-vitest-stable.mjs`, `scripts/test-line-shard.mjs` and `scripts/ci/refresh-test-durations.mjs`. **Current behavior** Baseline: green `PR` runs [43437558012](https://github.com/paperclipai/paperclip/actions/runs/43437558012) and [43441184316](https://github.com/paperclipai/paperclip/actions/runs/43441184316), all on m7i-flex.xlarge (4 vCPU, 16 GB). - Server shards: 9.5 to 11.7 job minutes, 502 to 654 s test step, effective file concurrency 2.5 to 2.9 of 3 workers. Vitest reports `tests 61-68%, import 25-31%, transform 6-7%, setup 1%`. Each shard sums 1,410 to 1,765 s of file time. The longest file is 303 s (`deferred-execution-receipt-work.test.ts`), so the floor is total work divided by workers, not one file. - Chat shards: 9.3 to 9.6, 7.6 to 7.8 and 6.4 to 6.5 job minutes. Each job ran two partitions of `chat-channels.integration.test.ts`. The partitions were balanced by collected-case count, but one `it.each` line can run many cases. One of six partitions recorded 532 s of test time against a 377 s mean. **Proposed behavior** - Chat partitions keep each source line whole and balance by recorded time per collected entry. - Each chat job runs three partitions in place of two. - General server weights come from the server jobs of merged PR runs, which run files with the same concurrency as the lane they place. **Reason and benefit** The slowest job sets the CI wall-clock time for every pull request. The chat lane goes from a 9.3-9.6 minute maximum to 6.5-6.8 minutes. ## What Changed - `scripts/test-line-shard.mjs`: `partitionTestLines` takes optional recorded durations. With durations, it balances source lines by recorded time; entries without a record take the median. Without durations, the behavior is unchanged. - `scripts/chat-shard-durations.json` (new): recorded time per collected chat entry, from the CI profile records of PR run [43444854171](https://github.com/paperclipai/paperclip/actions/runs/43444854171). 926 of 928 collected entries are measured; the other two are the bogus collection entries that the launcher already drops. - `scripts/ci/refresh-chat-shard-durations.mjs` (new): builds that manifest from downloaded `ci-profile-*` artifacts. Each executed case maps to the collected `it`/`it.each` entry that registers it. A case without an entry fails the refresh. - `scripts/run-vitest-stable.mjs`: a sharded chat job runs three partitions in place of two, and the chat launcher passes the recorded durations. A chat job collects the suite once with `vitest list` in place of once per partition, and the per-partition filter checks run at the same time. Before, each partition ran two blocking collections of the 86,000-line file one after another. The server worker count stays at three. - `scripts/ci/refresh-test-durations.mjs` and `scripts/ci/pr-run-durations.mjs`: the weekly refresh reads general server weights from the two newest merged-PR runs. Each run must have the full server shard count from `.github/ci-task-matrix.json` and cover at least 90% of the lane. Serialized weights still come from master Cloud readiness runs. - `scripts/general-server-shard-durations.json`: refreshed with the new source (PR runs 43439145752 and 43439279537). - Tests: `scripts/__tests__/run-vitest-stable-shard.test.mjs`, `scripts/ci/test-selection-pr-timings.test.mjs` and `server/src/__tests__/vitest-chat-shards.test.ts` cover the weighted partition, the entry mapping, the manifest format, the PR-run collection rules and the three-partition chat launcher. - `doc/DEVELOPING.md`: describes the three chat partitions, the chat weights and how to refresh them, and the new source of the server weights. ## Verification Experiments ran in this PR's own CI with a temporary resource sampler (removed before review). | Setting (run) | Server step vs. same shard's baseline | CPU busy | Max memory used | Result | |---|---|---|---|---| | 3 workers (43444854171, shards 1-4) | +8% (profiling overhead) | 62-73% | 6.8 GB | 1 flaky retry | | 4 workers (43444854171, shards 5-8) | -4% | 82-86% | 8.2 GB | clean | | 5 workers (43444854171, shards 9-12) | -10% | 70-93% | 9.5 GB | 1 hard failure: `decision-models.test.ts` timed out at 15 s twice | | 4 workers (43450469416) | mixed m5a/m5n/m6i runners | 78-89% | 7.7 GB | 1 hard failure: `workspace-recovery-warm-handoff.test.ts` timed out at 30 s on m5a | The server lane is CPU bound: at four workers the CPU is 82-89% busy and wall time improves by only about 10%. Five workers fail tests. Memory is not the limit (no swap, no memory pressure, at least 6 GB free). Node's compile cache and Postgres `fsync=off` showed no attributable gain (import share stayed 29-34%; `fsync=off` cut iowait from 2-4% to 0.2-0.7% of CPU time), so neither is in this PR. Final configuration (server: 3 workers with refreshed weights; chat: 3 weighted partitions per job), job minutes / test-step seconds / effective concurrency: | Shard | Baseline 43437558012 | Baseline 43441184316 | A: 43465156503 attempt 1 | B: attempt 2 | C: attempt 3 | |---|---|---|---|---|---| | server 1/12 | 11.0 / 609 / 2.81 | 10.2 / 562 / 2.82 | 10.1 / 535 / 2.88 | 11.6 / 615 / 2.86 | 11.3 / 600 / 2.86 | | server 2/12 | 10.2 / 567 / 2.71 | 10.9 / 607 / 2.79 | 9.7 / 510 / 2.85 | 8.9 / 466 / 2.86 | 11.9 / 635 / 2.83 | | server 3/12 | 10.9 / 609 / 2.88 | 10.9 / 601 / 2.86 | 9.4 / 498 / 2.81 | 11.7 / 623 / 2.80 | 20.3 / 1109 / 0.00 | | server 4/12 | 11.0 / 613 / 2.77 | 11.2 / 622 / 2.84 | 11.3 / 602 / 2.84 | 9.3 / 487 / 2.83 | 12.5 / 676 / 2.84 | | server 5/12 | 10.1 / 555 / 2.83 | 9.9 / 544 / 2.84 | 10.2 / 542 / 2.85 | 11.6 / 618 / 2.82 | 11.9 / 640 / 2.84 | | server 6/12 | 9.9 / 551 / 2.85 | 10.4 / 579 / 2.85 | 9.8 / 516 / 2.69 | 9.1 / 480 / 2.68 | 9.2 / 480 / 2.67 | | server 7/12 | 10.8 / 596 / 2.84 | 10.1 / 558 / 2.86 | 10.8 / 575 / 2.80 | 11.8 / 631 / 2.80 | 20.8 / 1129 / 2.69 | | server 8/12 | 11.7 / 654 / 2.62 | 11.2 / 624 / 2.56 | 9.8 / 513 / 2.83 | 11.2 / 589 / 2.82 | 19.4 / 1059 / 2.40 | | server 9/12 | 9.7 / 533 / 2.85 | 9.7 / 533 / 2.84 | 11.5 / 615 / 2.68 | 9.8 / 515 / 2.66 | 13.0 / 703 / 2.68 | | server 10/12 | 9.8 / 540 / 2.79 | 9.5 / 502 / 2.81 | 10.6 / 561 / 2.84 | 11.8 / 625 / 2.83 | 9.3 / 488 / 2.83 | | server 11/12 | 10.9 / 605 / 2.83 | 11.3 / 627 / 2.75 | 10.3 / 545 / 2.75 | 9.5 / 498 / 2.72 | 20.3 / 1107 / 0.00 | | server 12/12 | 10.6 / 592 / 2.50 | 11.7 / 645 / 2.53 | 9.6 / 503 / 2.81 | 8.9 / 468 / 2.82 | 19.7 / 1065 / 2.70 | | chat 1/3 | 9.6 / 528 / 1.56 | 9.3 / 511 / 1.58 | 6.6 / 310 / 2.69 | 5.7 / 272 / 2.65 | 6.8 / 331 / 2.69 | | chat 2/3 | 7.8 / 418 / 1.74 | 7.6 / 408 / 1.75 | 6.3 / 305 / 2.57 | 6.5 / 311 / 2.58 | 5.5 / 260 / 2.58 | | chat 3/3 | 6.5 / 337 / 1.74 | 6.4 / 333 / 1.73 | 6.0 / 285 / 2.65 | 5.4 / 258 / 2.68 | 5.8 / 262 / 2.69 | Summary: chat maximum 9.3-9.6 min before, 6.6 / 6.5 / 6.8 min in A / B / C. Server maximum 11.7 min before, 11.5 / 11.8 min in A / B; the server worker setting is unchanged, so its run-to-run spread is mostly runner type (m8i.xlarge shards took 8.9-9.8 min, m6a.xlarge 11.9-13.0 min). Runners: A all m7i.xlarge; B m8i/m6i; C m6a/m6in/m8i/m5a. In C, the five server shards on m5a.xlarge summed 2,545-3,037 s of file time against about 1,700 s elsewhere; three reached the 20-minute job limit and two failed tests on 15 s timeouts. The server lane settings in this PR are the same as master's, so this is a fleet finding: m5a.xlarge cannot run a server shard inside the job limit (follow-up for the fleet configuration, outside this PR). Flakiness of the final setting: the nine chat jobs in A, B and C had no failure and no retried test. Server shards in A and B had no failure and no retried test. Before the last commit (single chat collection), three runs of 43456289623 had chat at 6.7-7.5 min with no chat retries; server retries there (2 in one run, 1 in another) were on the unchanged three-worker server setting. Local checks (correctness only): - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/ci/test-selection-pr-timings.test.mjs scripts/ci/test-selection-timings.test.mjs`: 68 passed. - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/vitest-chat-shards.test.ts`: passed. - `pnpm verify` exits 2 (global CI input changed), so the required CI checks supply the broad coverage. ## Risks - Three chat partitions per job use more memory: the peak was 11.4 GB of 16 GB, with at least 4.2 GB free and no memory pressure. CPU was 55-63% busy. - Stale chat weights only change balance, never which tests run. Shard membership is still checked with `vitest list` before each run. - The weekly refresh now downloads general server logs from merged-PR runs. Runs that selected a subset of files are skipped before download because their shard count is smaller. - The server lane is still above 7 minutes. See the remaining floor below. **Remaining floor and shard-count recommendation (for a follow-up under `.github/`)** A server shard sums about 1,600 s of file time, and 3 workers on 4 vCPUs give at most about 2.8 effective concurrency, so the test step is about 560-640 s. Job overhead is about 50 s. A job of 7 minutes needs a test step of about 360 s, so about 1,000 s of file time per shard. The current 19,200 s of general server file time therefore needs about 19-20 shards of 4 vCPUs at 3 workers. More workers on the same runners do not help, because the CPU is already the limit. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, GitHub CLI) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0199c963e2 |
feat(apps): ship the redesigned sign-in step for a pasted MCP server (#21469)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Apps lets an operator connect a remote MCP server by pasting its
address ("Connect your own MCP server")
> - Step 2 of that flow put every sign-in choice in an "Advanced
authentication" box with wrapped pill buttons and a hint paragraph
> - The box hid the choice behind a vague label, and its shape did not
match the access line ("Connects for everyone… Change") right below it
> - The owner approved a redesign: a stated sign-in default with a
Change link, and five options with their fields directly below each
option
> - This pull request wires that redesign into the real screen, removes
the old step, and updates copy that named the removed box
> - The benefit is a screen that reads as two stated answers and one
action, with the same default path (paste the address, press Check link)
## Linked Issues or Issue Description
No public issue exists. Enhancement:
**What existing behavior does this improve?**
The sign-in section of the "Connect your own MCP server" step in Apps.
**Subsystem affected**
UI: Apps connection setup (`ui/src/features/connections`). Small copy
changes in the server OAuth client registration message and docs.
**Current behavior**
Step 2 shows an "Advanced authentication" collapsible with five wrapped
pill buttons and a hint paragraph below them. The fields for the chosen
mode appear further down. Guidance banners tell the operator to look
"under Advanced authentication". The page also shows the "Unverified
server" badge twice (header and body). The stepper reads "Pick app · Add
your key", even when the server needs no key.
**Proposed behavior**
Step 2 states the sign-in choice in one line ("Paperclip asks the server
how to sign in.") with a Change link. Change shows five radio cards: Let
Paperclip check, Key or token, Browser sign-in, Custom headers, No
sign-in needed. The fields for the selected option are directly below
it. Done collapses the list to a one-line summary (for example "Signs in
with 1 custom header."). The list stays open while the chosen option
still needs a value. Client ID and secret are behind "Use your own
client ID" unless they have values or the server asked for them. The
stepper reads "Pick app · Connect".
**Reason and benefit**
The choice is visible as a stated default in the same shape as the
access line. Each option keeps its fields next to it. No copy points at
a box that does not exist.
**Breaking changes**
None. The five auth modes and their request payloads do not change.
**Additional context**
Related open PRs change server behavior for the same flow. They are not
duplicates. Refs #15192, Refs #15450.
## What Changed
- `ui/src/features/connections/GenericMcpConnectStep.tsx`: new step
component (same props as the old `LinkConnectStep`). It uses stable
field ids `generic-mcp-key`, `generic-mcp-client-id`,
`generic-mcp-client-secret` and a `data-testid` on the sign-in
Change/Done link.
- `ui/src/features/connections/ConnectionSetupFlow.tsx`: render
`GenericMcpConnectStep`; delete `LinkConnectStep`,
`GENERIC_AUTH_MODE_OPTIONS`, `SegmentedOption` and unused imports;
change the pasted-link stepper label from "Add your key" to "Connect"
(`STEP_LABELS` is used only by the pasted-link flow; curated apps keep
their own `["Add your key"]`).
- `ui/src/pages/apps/generic-mcp-connect.ts`: `oauth_challenge` and
`oauth_manual_client_required` guidance now say "below" instead of
"under Advanced authentication".
- `server/src/services/tool-access/oauth/client-registration.ts`: the
manual-client message no longer names the removed box.
- `doc/connections/GENERIC-REMOTE-MCP.md`: describe the sign-in line,
Change, and the five options.
- Comments in `packages/shared` and `server/src/services/tool-access.ts`
reworded. `tests/runner-e2e/source-sdk-public-inputs.json` regenerated
for the two shared files (blob hashes only).
- `ui/storybook/stories/connection-setup/byo-mcp-sign-in.stories.tsx`:
14 stories for the shipped component; annotation is now `current` /
`production`.
- Tests: `AppsConnect.test.tsx` generic MCP block uses the sign-in
toggle and radio options; new tests for the collapsed summary, the
open-while-empty-key rule, and a single "Unverified server" label. New
`GenericMcpConnectStep.test.ts` unit-tests `genericMcpAuthSummary`.
Before: "Advanced authentication" box with pill buttons and a hint;
fields below the pills; two "Unverified server" badges; stepper "Pick
app · Add your key".
After: one sign-in line with Change; five radio cards with fields under
the selected card; one "Unverified server" badge in the header; stepper
"Pick app · Connect". Storybook: `byo-mcp-sign-in--choosing`,
`byo-mcp-sign-in--server-asked-for-key`.
## Verification
- `cd ui && pnpm exec vitest run src/pages/apps/AppsConnect.test.tsx
src/pages/apps/generic-mcp-connect.test.ts
src/features/connections/GenericMcpConnectStep.test.ts
src/features/connections/ConnectionSetupFlow.architecture.test.ts
src/features/connections/ConnectionSetupFlow.oauth.test.ts`: 5 files,
300 tests pass.
- `cd ui && node ../scripts/run-tsc.mjs -b`: pass.
- `pnpm -s typecheck` in `packages/shared` and `server`: pass.
- `pnpm check:token-gates`: all gates clean.
- `node scripts/storybook-catalog.mjs`: pass (generated indexes not
committed).
- `pnpm verify`: exit 2 (broader verification required, because
`doc/connections/GENERIC-REMOTE-MCP.md` is a global CI input). Required
CI supplies that coverage. The focused files above were rerun on the
final revision after the rebase.
- Headless Playwright against the Storybook dev server: the Default,
Choosing and Server-asked-for-key stories render the new step, the new
guidance copy, and the "Pick app · Connect" stepper.
## Risks
- Low risk. UI change on one screen; request payloads
(`genericConnectPayload`) do not change.
- Tests that click the first collapsed "Change" button now skip the
sign-in toggle, because the access line also has a "Change" button.
- The server message for `oauth_manual_client_required` changes text. No
code matches on the text; the client uses the error code.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, headless Playwright).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
|
||
|
|
e9d7895648 |
fix(recovery): start an overdue saved-work recovery instead of looping on no pending path (#21478)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - When a run stops before its sandbox files are saved, Paperclip holds
the task and recovers the files first. A recovery queue does this work.
> - The queue runs two attempts at a time for the whole instance. A
remote attempt can take many minutes.
> - On staging, 29 recoveries waited with 0 attempts. The oldest waited
since 15:01Z. One task waited 3 h 33 min for its single attempt.
> - During that time the task said "Recovering saved work…", but the
assigned-work invariant saw no pending path. The sweep sent a
continuation wake every few minutes. The hold refused each wake, and the
activity log showed "could not start yet".
> - This pull request makes the queued recovery the task's pending path
for a bound. After the bound, the sweep starts the recovery itself
instead of sending a wake that the hold refuses.
> - The benefit: a held task gets its files recovered and continues
without a person, and the sweep stops its refused-wake loop.
## Linked Issues or Issue Description
**What happened?**
A scheduled task ran a legacy retry on a Daytona sandbox. Setup failed
with `setup_failed` ("remote_workspace_operation_timed_out: capture
(configured timeout 300000ms)"). Paperclip held the task for saved-work
recovery (`legacy_execution_requires_reconciliation`, resolved, replay
blocked, system recovery `scheduled`, 0 attempts). After 45 minutes the
recovery had not started. The task showed "Recovering saved work…" with
no Retry. The assigned-work sweep logged `issue.continued_automatically`
with outcome `not_started` and violation `no_pending_path` at 17:58,
18:02, 18:10 and 18:25 UTC.
**Expected behavior**
The recovery proceeds by itself and the task continues. If the recovery
cannot finish, the task shows a Retry that works. "Recovering…" never
shows with nothing behind it.
**Steps to reproduce**
1. A legacy run with a retained remote workspace fails during setup, and
its saved-work hold is adopted by system recovery.
2. The recovery queue does not reach the hold (it has two slots, and
they are busy).
3. The assigned-work sweep checks the task 10 or more minutes later. It
finds `no_pending_path` and sends a wake. The hold refuses the wake.
This repeats until the continuation cap.
The regression test
`server/src/services/saved-work-recovery-overdue.test.ts` reproduces
this state on real rows.
**Paperclip version or commit**
`master` at `c6301c92ff` (staging, 2026-11-13).
**Agent adapter(s) involved**
Not adapter-specific (core bug). Seen with a Daytona sandbox on the
legacy runtime.
## What Changed
- `assigned-work-invariant.ts`: new pending intent
`saved_work_recovery`. A saved-work recovery that the queue still owes
counts as the task's pending path while an attempt runs, while its next
attempt is scheduled, or for `SAVED_WORK_RECOVERY_START_BOUND_MS` (10
minutes) after it became due.
- `assigned-work-invariant.ts`: new violation
`saved_work_recovery_overdue`, used when that bound passes. It comes
before the `no_pending_path`, `retry_offered` and blocked checks, so the
sweep never sends a wake that the hold refuses.
- `assigned-work-sweep.ts`: the sweep loads the fact with the same
eligibility predicate the recovery worker uses. For the overdue
violation it calls `startSavedWorkRecovery` and records no continuation
activity.
- `workspace-recovery.ts`: `startOverdueWorkspaceRecovery` starts one
attempt outside the queue's two slots, with at most two such attempts
per process. It skips runs that are already in flight and uses the same
ownership lock and attempt bookkeeping as the queue. The queue skips
runs that this path started. `recoverRunAction` moved to module scope.
Its "Retained workspace remains pending" warning now logs the error.
- `heartbeat.ts` and `recovery/service.ts`: wire the dependency.
- The sweep returns the end of the bound as the task's re-check
deadline, so the overdue start does not depend on another notification.
## Verification
- New `server/src/services/saved-work-recovery-overdue.test.ts` (real
Postgres, recovery fixture with a mocked sandbox):
- Overdue hold: the sweep starts the recovery and does not call
`enqueueWakeup`. No continuation activity is written. The system
recovery reaches `complete`. The next sweep records a fresh
continuation, and `deliverReconciledExecutions` starts a successor run.
- Hold within the bound: verdict `ok / pending_intent /
saved_work_recovery`. No wake is sent and no attempt starts.
- A second start for the same run returns `in_flight`. Exactly one
attempt is recorded, and a completed hold returns `not_eligible`.
- Red on the old invariant and sweep, with `no_pending_path` and
`enqueue_refused`, the same as staging. Green with this change.
- `server/src/services/recovery/assigned-work-invariant.test.ts`: 54
pass, with new cases for the bound, a running or scheduled attempt, a
stale attempt, and a pending question.
- `server/src/__tests__/assigned-work-invariant.test.ts`: 31 pass.
- `server/src/services/workspace-recovery-passive-fairness.test.ts`:
passes.
- `tsc --noEmit -p server/tsconfig.json`: clean.
## Risks
- A sweep-started attempt can run beside the queue's two attempts. That
adds up to two more concurrent recovery attempts per process, each with
its own database connection while it holds the ownership lock. Each
attempt for a run still takes that run's advisory lock, so two servers
cannot run the same recovery at once.
- The 10-minute bound is new. A queue that is slow but working now gets
help after 10 minutes. It does not get the hours it took on staging.
- Out of scope, reported for follow-up: why the queue was so slow on
staging (two slots for the whole instance, attempts that take minutes),
and why the setup capture hit its 300 s timeout. Server logs were not
available from this session.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, browser reads of the staging API, local test
runs).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
f58bbefc4e |
fix(ui): group failed-query Sentry reports by status and query name (#21475)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The browser app can report errors to Sentry. A final query error goes to Sentry when it is not an outage, a denial, or a missing item. > - Every failed API request throws an `ApiError` from the same frames in `api/client.ts`. > - Sentry groups errors by stack. Thus all unrelated server failures go into one issue, titled "ApiError: Internal server error". > - The report has no route, no query name, and no status. Breadcrumbs and URLs are removed for privacy. A person cannot find which endpoint failed. > - This pull request tags each failed-query report with its HTTP status and the static name of its query key. It also sets the Sentry fingerprint from these two values. > - The benefit: each failing endpoint gets its own Sentry issue, and a person can connect it to the server error. ## Linked Issues or Issue Description **What happened?** One browser Sentry issue collected unrelated server 500 errors from different pages and environments. The events had no route or query name. A person could not connect an event to a server error. **Steps to reproduce** 1. Turn on browser error monitoring with a Sentry DSN. 2. Make two different queries fail with a 500 response, for example the issues list and the agents list. 3. Look at Sentry. Both events are in one issue with the title "ApiError: Internal server error". Neither event shows which query failed. **Expected behavior** Each failing query has its own Sentry issue. The event shows the HTTP status and the static query name. It shows no IDs, refs, or filters. **Environment** The hosted browser app, staging and production builds. I searched open pull requests for duplicates. I found no pull request that changes browser query-error grouping. ## What Changed - `ui/src/lib/browser-error-context.ts`: add `queryKindFor`. It keeps only the first query-key part, and only when it is a lowercase word list such as `issues` or `agent-commentary`. Other values become `other`, so an ID, a ref, an email, or a typed value is never sent. - `ui/src/lib/browser-error-context.ts`: add `buildQueryErrorContext`. For an error with an HTTP status, it sets the `query_kind` and `api_status` tags and the fingerprint `["api-error", status, queryKind]`. For an error without a status, it sets only the `query_kind` tag and keeps default stack grouping. - `ui/src/lib/sentry.ts`: add `captureQueryException`. It uses the same serialized queue as `captureBrowserException`. Both functions now share one private capture step. - `ui/src/lib/query-client.ts`: `reportError` now also receives the query key. - `ui/src/main.tsx`: the query client reports through `captureQueryException`. - Tests for key sanitizing, the fingerprint, the reporter argument, and the gate call. ## Verification - Red before the fix: the 6 new assertions fail on the old source (`queryKindFor`, `buildQueryErrorContext` and `captureQueryException` do not exist, and `reportError` gets no key). - Green after the fix: `cd ui && npx vitest run src/lib/sentry.test.ts src/lib/browser-error-context.test.ts src/lib/query-client.test.ts` passes 3 files and 78 tests. - The privacy test makes sure that `company-uuid` and `PAP-1` from later key parts do not go into the Sentry call. It also makes sure that UUID, ref, email, and long heads become `other`. - `ui` typecheck: no errors in the changed files. The local install was filtered to the UI package, so unrelated packages (`plugin-sdk` build output, runner dependencies) could not resolve. Required CI supplies the full typecheck. - I did not run browser e2e suites. This change does not touch them. ## Risks - Low risk. Only the grouping and tags of browser query-error reports change. The app does not change what it shows to users. - Sentry opens new issues for future query errors, one for each status and query name. The old catch-all issue gets no new events. - A query key with a dynamic first part becomes `other`. Those errors still group together, but they keep the status tag. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in Claude Code with tool use, repository inspection, and shell code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ef2a49b6df |
fix(daytona): bound the duplex channel close so a stalled kill cannot retire the worker (#21471)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cloud agent runs use the Daytona sandbox provider plugin. One shared plugin worker process serves all Daytona calls on a server. > - The host closes a duplex channel through the worker. The host waits 10 seconds for the close acknowledgement. When the acknowledgement is late, the host retires the whole worker with SIGKILL. > - The worker close waits for the remote PTY kill before it releases the socket. The Daytona SDK sends that kill as a toolbox request with a 24-hour default timeout. A stalled sandbox can hold it open for longer than 10 seconds. > - Then the host kills the shared worker. Every other call in flight on that worker fails with `Worker process exited (code=null, signal=SIGKILL)`. This includes other runs' workspace preparation and restore. > - This pull request limits the wait for the remote kill to 5 seconds. Then the worker releases the socket and acknowledges the close. > - The benefit is that one slow sandbox no longer stops unrelated runs on the same server. ## Linked Issues or Issue Description **What happened?** On a hosted staging server, the host logged `duplex channel close not acknowledged; retiring worker` six times in about 90 minutes. Each time, the Daytona worker exited with SIGKILL. Each time, all in-flight calls on that worker failed. Runs in workspace preparation failed with `setup_failed` and `Worker process exited (code=null, signal=SIGKILL)`. Several runs failed together in each burst. **Expected behavior** A slow close of one duplex channel does not stop calls that belong to other channels or other runs. **Steps to reproduce** 1. Open a duplex channel on a Daytona lease. 2. Make the Daytona PTY kill request stall (the SDK default request timeout is 24 hours). 3. Close the channel from the host. The worker does not acknowledge within 10 seconds. 4. The host retires the worker with SIGKILL. All other pending calls fail. **Relevant logs or output** ``` info duplex channel close not acknowledged; retiring worker info retired worker process exited code=null signal=SIGKILL willRestart=true JsonRpcCallError: Worker process exited (code=null, signal=SIGKILL) ``` **Additional context** The login PTY close does not have this problem. It only does the local socket disconnect. A kill that rejects already gets a close acknowledgement, because the session close ignores kill errors. This change gives a slow kill the same result. No related public PR exists. I searched open and closed PRs for this failure. ## What Changed - `duplex-command-stream.ts`: `close()` waits for the remote `kill()` for at most `DUPLEX_CHANNEL_CLOSE_KILL_TIMEOUT_MS` (5 seconds). Then it always releases the socket with `disconnect()`. The kill continues in the background. A kill that answers in time keeps the old order: kill, then disconnect, then the acknowledgement. - `duplex-close-deadline.test.ts` (new): regression tests at the session close and at the worker `onDuplexChannelClose` boundary, and a check that the deadline stays well inside the host deadline. ## Verification - Red before the fix: with the old `duplex-command-stream.ts`, `acknowledges a bound close inside the host deadline when the remote kill never settles` fails (no acknowledgement after 9 seconds), and the session close test times out. - Green after the fix: `pnpm --dir packages/plugins/sandbox-providers/daytona exec vitest run --config vitest.config.ts src/duplex-close-deadline.test.ts src/duplex-command-stream.test.ts src/plugin.test.ts` passes 453 of 453. This includes the existing tests `joins pending teardown for concurrent closes while all channel authority stays revoked` and `does not turn a rejected retired close into an acknowledgement`. - The safety case `still waits for a kill that answers inside the deadline before it releases the socket` passes before and after the fix. - `tsc --noEmit` for the Daytona plugin passes. - In the full Daytona plugin suite, two GNU tar tests in `file-sync.test.ts` fail in my sandbox. They also fail on unchanged `master` in the same sandbox, so they are environment failures. CI covers them. - I did not run a live staging workflow. After the deploy, the staging logs must show no new `duplex channel close not acknowledged; retiring worker` lines from slow kills. ## Risks - Low risk. The change is in one Daytona plugin function. - After a kill timeout, the gateway process can stay alive in the sandbox until the background kill completes or the lease teardown stops the sandbox. The socket is closed, so no bytes can move on the channel. The route stays retired, so the host route cannot open again. This is the same state as when a kill rejects today. - The host contract does not change. A malformed or mismatched acknowledgement still retires the worker. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> |
||
|
|
7773b52f05 |
fix(codex): preserve sandbox certificate trust in Codex shell commands (#20364)
<!-- Written in ASD-STE100 style: short sentences, active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Paperclip Runner and the Codex adapters run agent shell commands inside a sandboxed Codex session with a strict environment allowlist > - The controller builds `shell_environment_policy.include_only` before the sandbox injects certificate-trust variables such as `SSL_CERT_FILE` and `GIT_SSL_CAINFO` > - The allowlist drops the injected variables, so managed `git` and `gh` commands cannot verify HTTPS against a runtime-provided CA > - This pull request allows the certificate-trust variable names through every environment policy stage, from the Rust process supervisor to the shell allowlist > - The benefit is that managed `git` and `gh` work again in sandboxes that require a runtime CA, while provider-credential isolation and the bounded allowlist stay in place ## Linked Issues or Issue Description - Fixes: #20019 - Refs #20038 — community pull request for the same defect. This change reviews the approach independently. The commit credits the community author as co-author. ## What Changed - Add `CERTIFICATE_ENVIRONMENT_KEYS` (`SSL_CERT_FILE`, `SSL_CERT_DIR`, `GIT_SSL_CAINFO`, `NODE_EXTRA_CA_CERTS`, `DENO_CERT`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`) to `packages/paperclip-runner/src/configured-environment.ts` - Native Codex shell policy (`codex-security-config.ts`): always put the certificate names in `include_only`; set inheritance to `"all"` with `ignore_default_excludes=true`, bounded by the explicit allowlist - Pass the certificate names through the app-server environment filter, the ACPX spawn environment filter, the ACPX runtime sandbox `include_only`, the runner platform environment keys, and the Rust process supervisor base key list - Legacy and ACPX-engine Codex shell allowlists (`packages/adapter-utils/src/codex-shell-environment.ts`): always include the certificate names - Add a synthetic private-CA HTTPS journey test: a real `git ls-remote` fails TLS verification without the injected CA and succeeds with it, through the policy-filtered child environment - Document the contract in `doc/sandbox-shell-environment.md` and link it from `doc/DEVELOPING.md` ## Verification - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/drivers/codex/codex-security-config.test.ts src/configured-environment.test.ts src/drivers/codex/codex-shell-certificate-trust.test.ts src/drivers/codex/codex-app-server-driver.context-control.test.ts src/drivers/acpx/runtime-sandbox.test.ts` — 104 tests pass - `pnpm --filter @paperclipai/adapter-utils exec vitest run src/codex-shell-environment.test.ts src/acpx-engine/execute.test.ts` — 270 tests pass - `pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/execute.test.ts` — 10 tests pass - Codex driver lifecycle, conformance, recovery, and boundaries suites pass (103 tests); environment-builder suites pass (307 tests) - Red proof: with only the source fix reverted, the new regressions fail. The journey test shows `GIT_SSL_CAINFO` is undefined in the shell environment, and `git` fails TLS verification against the private-CA fixture - `cargo check` passes for `paperclip-runner-core`; `rustfmt --check` passes for the edited Rust file - The journey test needs `git` and `openssl`; it skips when either tool is absent ## Risks - The native shell policy now always sets `shell_environment_policy.inherit="all"` with `ignore_default_excludes=true`. Before, it set `"none"` when no credentials or task keys were projected. The explicit `include_only` allowlist still bounds every inherited name, so no new secret names can cross - The allowlist grows by seven non-secret certificate-trust names. Their values point at CA files. The sandbox filesystem policy still controls access to those files; this change grants no filesystem access - No persisted run, session, or launch identity changes shape. Sessions saved by the previous release resume unchanged; the new policy applies when the next launch builds its arguments - Provider credentials keep their existing exclusions at every stage ## Model Used - Claude Fable 5 (Anthropic, model ID `claude-fable-5`), extended thinking, agentic tool use, run through Claude Code. The implementation was written and reviewed independently against the community design in #20038; the community author is credited as co-author in the commit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending: checks run after this PR opens) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending review) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: everlastingcrown <everlastingcrown@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>canary/v2026.1113.0-canary.11 |
||
|
|
d018a9d1cd |
fix(queue): retain chat bindings on resolved card continuations (#20825)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task can receive a card response while its agent runs.
> - The queue saves this response for the next turn.
> - Chat authorization requires the saved inbound message binding.
> - Queue promotion removed this binding when a prior turn received the
message.
> - This pull request retains live bindings for resolved card responses.
> - The next turn can check current chat access and continue the
approved work.
## Linked Issues or Issue Description
**What happened?**
An accepted plan card queued behind a chat run could start with a chat
source and no message IDs. Reviewed chat authorization then failed with
context_mismatch before provider work started.
**Expected behavior**
A resolved card must retain its live chat binding through queue
promotion. The new run must check current access. A consumed
message-only wake must not start another turn.
**Steps to reproduce**
1. Start a chat run with an inbound message.
2. Queue a card response with that message binding.
3. Complete the first run and release its queue.
4. Inspect the promoted context and check chat authorization.
Related work: #20689 cancels consumed message-only wakes. #19156
protects a successor checkout. This change retains the binding for an
independent card response.
## What Changed
- Retain delivered message bindings for resolved chat card responses
during queue promotion.
- Keep deleted, foreign-task, and self-authored message filters and
current chat authorization.
- Add a persisted queue regression and Slack/Discord
promotion-to-authorization checks, including access revocation.
- Document the queue behavior.
## Verification
- Before the fix, the persisted queue test failed because the promoted
context had no wakeCommentIds. Three existing tests passed.
- The first focused run passed 108 tests. The added Slack and Discord
checks passed after promotion and refused revoked access.
- Final head
|
||
|
|
b4861d5e37 |
fix(diagnostics): report Daytona acquisition state for runs retried as provider outages (#21476)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Before an agent starts on a sandbox environment, the host gets an environment lease. On Daytona, this creates a sandbox. > - When the create does not finish in time, the Daytona plugin records the failed phase, the elapsed time, the budget, and the Daytona sandbox state. #15736 and #15946 added these fields to the run failure report, so that operators can see where the acquisition stopped. > - #20857 changed the error code of these failures from `setup_failed` to `environment_provider_unavailable`, so that the host retries them. > - The run failure diagnostics read the acquisition fields only when the error code is `setup_failed`. Thus every Daytona create timeout now reaches the error report without its phase or provider state. > - On staging, more than 270 Daytona lease timeouts occurred in three days, and none of them shows the provider state. > - This pull request also reads the acquisition fields when the error code is `environment_provider_unavailable`. > - The benefit is that the error report shows again whether the sandbox was stuck in `creating`, in a snapshot state, or absent. This evidence is necessary to find the cause of the timeouts. ## Linked Issues or Issue Description Refs #20857, Refs #20828, Refs #15946, Refs #15736 **What happened?** A Daytona lease acquisition timeout is now classified as `environment_provider_unavailable` (#20857). The run failure report for this failure does not contain `environmentAcquisitionPhase`, `environmentAcquisitionElapsedMs`, `environmentAcquisitionBudgetMs` or `environmentAcquisitionProviderState`. **What did you expect?** The report contains the acquisition phase and the provider state, as it did for `setup_failed` runs before #20857. **Steps to reproduce** 1. Let a Daytona sandbox create pass its acquisition deadline. 2. The plugin returns "Daytona lease acquisition timed out; allocation cleanup is pending" with an acquisition diagnostic. 3. The host fails the run with `environment_provider_unavailable`. 4. Look at the `run_execution` context of the error report. The acquisition fields are not there. ## What Changed - `collectRunFailureDiagnostics` reads the private acquisition diagnostic for a setup failure with the error code `setup_failed` or `environment_provider_unavailable`. The other conditions do not change: the phase must be `setup`, and the diagnostic must belong to the same run and company. - A new test builds the real wrapper chain (lease failure, then provider failure, then the RPC error that holds the diagnostic). It checks that the fields are reported for the failed run, and that they are not reported for another run or for the execute phase. ## Verification - `cd server && npx vitest run src/services/__tests__/run-failure-diagnostics-provider-unavailable.test.ts` fails on master (the report has only `failurePhase`). It passes with this change. - `cd server && npx vitest run src/services/__tests__/run-failure-diagnostics-provider-unavailable.test.ts src/services/__tests__/run-failure-diagnostics.test.ts src/__tests__/environment-acquisition-diagnostics.test.ts`: 3 files, 139 tests pass. ## Risks Low risk. The change adds four bounded scalar fields to an error report. These fields already exist for `setup_failed` runs. The change does not alter retries, the error grouping, or the run result. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in an agent harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
|
89ab7e51ff |
fix(recovery): retain a failed run's sandbox in one lease write (#21474)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs execute in remote sandboxes. Each run holds an
environment lease row that points to its environment, task, workspace
and company
> - When a run fails before it copies its files back, Paperclip keeps
the sandbox for a file repair. It writes a stop-and-retain intent and a
repair-hold policy on the lease
> - The recorder wrote the same lease row two times in one transaction.
Postgres runs the foreign-key checks again on a row that the current
transaction already wrote, and these checks take `FOR KEY SHARE` locks
on the referenced rows
> - Other runs lock the shared environment row `FOR UPDATE` when they
acquire a lease. Then the second write waits, meets the 1 second lock
timeout, and the run fails with "Database query failed (55P03)"
> - This pull request writes the stop intent and the repair hold in one
update
> - The benefit is that a busy shared environment no longer fails runs,
and the file repair hold is recorded
## Linked Issues or Issue Description
**What happened?**
On staging, native OpenCode runs failed with `adapter_failed` and the
error "Database query failed (55P03)". The failed statement was `update
"environment_leases" set "lease_policy" = 'retain_on_failure',
"metadata" = "metadata" || {workspaceRestoreRecovery...} where ...
"status" = 'pending_cleanup'` in
`preserveLegacyWorkspaceRestoreSources`. The cause was `canceling
statement due to lock timeout`. The statement does not wait on the lease
row, because the transaction already holds it `FOR UPDATE`. It waits on
the foreign-key check `SELECT 1 FROM ONLY "public"."environments" x
WHERE "id" = $1 FOR KEY SHARE OF x`. Postgres runs this check because
the transaction wrote the same row one statement before, in
`prepareSandboxStopAndRetain`.
**What did you expect?**
The run records its workspace restore failure and keeps its sandbox for
repair, also when another run holds the shared environment row.
**Steps to reproduce**
1. Start a run with an active, ephemeral sandbox lease.
2. In another session, lock the lease's `environments` row with `SELECT
... FOR UPDATE` and keep the transaction open.
3. Record a workspace restore failure for the run
(`terminalizeLegacyExecution`, legacy or native
`recordRestoreFailureOnly`).
4. Before this change, the second lease write fails after 1 second with
55P03. The new test
`server/src/__tests__/legacy-workspace-restore-lock-contention.test.ts`
shows this for legacy and native runs.
No public issue exists. I searched open and recent pull requests for
this lock timeout and found no duplicate. #21071 bounds a different lock
wait (workspace source locks in run preparation) and does not overlap.
## What Changed
- `server/src/services/sandbox-stop-and-retain.ts`:
`prepareSandboxStopAndRetain` accepts an optional `retention` argument
(`leasePolicy` and extra `metadata`). The stop-and-retain update sets
these values in the same statement. Other callers do not change.
- `server/src/services/legacy-workspace-restore-recovery.ts`: the
active-source path passes the repair hold (`retain_on_failure` and
`workspaceRestoreRecovery` with `originalLeasePolicy`) to that update.
It no longer does a second update on the same row.
-
`server/src/__tests__/legacy-workspace-restore-lock-contention.test.ts`:
a regression test for legacy and native runs. Another session holds the
environment row `FOR UPDATE` while the run records its restore failure.
The test checks the run's recovery evidence and the lease policy,
status, stop intent and repair metadata.
## Verification
- `cd server && npx vitest run
src/__tests__/legacy-workspace-restore-lock-contention.test.ts`: both
cases fail on `master` with `canceling statement due to lock timeout` on
the `environments` `FOR KEY SHARE` check. Both cases pass with this
change.
- `cd server && npx vitest run
src/__tests__/legacy-workspace-restore-recovery.test.ts`: 73 existing
tests pass.
- Related suites pass (10 files, 485 tests):
`heartbeat-observed-workspace-preservation`,
`heartbeat-native-remote-failure-retention`, `live-recovery-work`,
`workspace-recovery-stop-proof`, `native-failed-workspace-recovery`,
`native-cancelled-workspace-preservation`,
`workspace-recovery-admission`, `native-workspace-reconstruction`,
`environment-retained-orphan-sandbox`, `abandoned-workspace-files-hold`.
- I did not complete a local server typecheck. The process ran out of
memory in the 10 GB sandbox. CI runs the typecheck on this head.
## Risks
- Low risk. The lease ends with the same status, policy and metadata as
before. The values are written in one statement instead of two. The
update keeps its ownership guards (status, provider, provider lease ID
and exact metadata).
- Other code paths can still write a lease row two times in one
transaction (for example the shutdown settlement path, which sets a
lease back to `active` before it records the restore failure). That path
handles 55P03 as a lock timeout and retries. This pull request does not
change it.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, code execution, embedded Postgres tests, and
GitHub tools).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
7866e9ef05 |
fix(acpx): admit a Claude tool call only for a tool use its turn announced (#21278)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner runs Claude Code through the ACPX sidecar. Claude calls Paperclip tools through a Runner-owned MCP endpoint, the tool bridge. > - The bridge identifies a call by its JSON-RPC request id. That id does not say which turn sent the call. The sidecar binds a call to the turn that is active when the call arrives. > - Because of this, the Runner stops and starts the Claude process at each turn boundary. A late call from an earlier turn then cannot run in a later turn. This restart also stops all background work that Claude started. > - Claude Code already sends the missing identity. Each MCP `tools/call` has `_meta["claudecode/toolUseId"]`. It is the id of the ACP `tool_call` that the provider announces first, inside one turn. > - This pull request uses that id. The sidecar admits a Claude tool call only if the active turn announced its tool use. > - The benefit is that a call from an earlier turn is refused by its identity. This is the first step to keep the Claude process alive between turns. This pull request does not change the restart. ## Linked Issues or Issue Description No public issue exists. The issue is described here. Refs #19912 (the design for turn-scoped ACPX MCP authority), #12419 (the restart at turn start) and #20190 (the restart at turn end). This pull request is the bottom of a stack. #21274 is merged, so the base branch is `master`. It uses no code from #21274. The stack order lets a later pull request remove the direct gateway path after both. **What existing behavior does this improve?** How the Runner decides which turn a Claude tool call belongs to. **Subsystem affected** Paperclip Runner: the tool bridge and the ACPX sidecar, for the Claude provider only. **Current behavior** - The bridge keys a call by the JSON-RPC request id (`runner-tool-bridge.ts`). - The sidecar gives the call to the active turn (`waitForTool` in `acpx-runtime-sidecar.ts`). - After `session/cancel`, Claude Code does not stop an MCP request that is in flight. A test run showed two calls from two turns live at the same time on one MCP session. Request ids cannot tell them apart. **Proposed behavior** - The bridge can read a provider tool-use id from `params._meta` and give it to the handler. It keys replays by that id. - For Claude, the sidecar admits a call only when the turn event stream of the active turn contains the ACP `tool_call` with the same id, and no earlier call used that id. - The announcement and the call use different channels. If the call arrives first, the sidecar waits for the announcement for a maximum of 5 seconds. The wait stops when the turn ends or the call is cancelled. - A call with no id, an id from an earlier turn, an id that was never announced, or an id that was used before gets a tool error. There is no fallback. - The call id that the sidecar sends to runnerd is `tool-use:<id>` for Claude. **Reason and benefit** - A late call from an earlier turn cannot run in a later turn. The identity of the call prevents this, not only the process restart. - A later change can stop the restart between turns. ## What Changed - `packages/paperclip-runner/src/drivers/runner-tool-bridge.ts` - New option `providerToolUseIdMetaKey`. Without it the bridge does not change. - With it, the bridge reads `params._meta[key]`. It refuses a value that is not a bounded string. It gives a valid value to the handler as `providerToolUseId`. - The replay key is the tool-use id. A retry with a new request id replays the first result. The same id with different arguments is refused. - `notifications/cancelled` still finds the call by its request id. - `packages/paperclip-runner/src/drivers/acpx/claude-tool-use-admission.ts` (new): `ClaudeToolUseAdmission`. - `observe` records an id only from an event with `type === "tool_call"` and `tag === "tool_call"`. A `tool_call_update` does not count, so a late update about an old tool use cannot admit it. - `admit` applies the rule above. An id is used when it is admitted. Limits: 4,096 announcements for each turn and 8,192 admitted ids. At a limit, new calls are refused. - `packages/paperclip-runner/src/cli/acpx-runtime-sidecar.ts` - The sidecar creates the admission object only for `agent === "claude"` and sets the bridge option only for Claude. - `pumpTurn` gives each turn event to `observe`. `waitForTool` calls `admit` and then checks the turn and run again. - The internal `get_task_context` call for a completion continuation does not go through admission. - The Claude call id does not use the per-process namespace, because a tool-use id is unique. - No change to Rust, the sidecar contract, the `claude-agent-acp` patch, the server, or restart, park and turn lifecycle code. Other ACPX agents and Cursor Cloud use the same code path as before. Why 5 seconds: the measured time between the announcement and the call is 1.1 ms to 14.7 ms. The limit must also include queue and event-loop delay. 5 seconds is far above that, and a call that cannot be attributed still gets a fast answer. ## Verification Commands, from `packages/paperclip-runner`: - `npx vitest run src/drivers/acpx/acpx-runtime-adapter.claude-tool-use.test.ts src/drivers/acpx/claude-tool-use-admission.test.ts src/drivers/runner-tool-bridge.test.ts src/drivers/runner-tool-bridge.claude-tool-use.test.ts src/drivers/runner-tool-bridge.claude-schema.test.ts src/cli/acpx-runtime-sidecar.test.ts src/drivers/acpx/acpx-runtime-adapter.test.ts src/drivers/cursor-cloud/cursor-cloud-driver.test.ts`: `8 passed (8)` files, `384 passed` tests. - `npx tsc --noEmit -p tsconfig.json`: no errors. New tests: - **Real ACP runtime** (`acpx-runtime-adapter.claude-tool-use.test.ts`). It uses the real `openCodexAcpxRuntime`, ACPX 0.22, the pinned and patched `claude-agent-acp`, the pinned Claude Code client and a real tool bridge, against a local fake Anthropic API. - Turn 1: two parallel tool uses are admitted as `tool-use:<id>`. The model receives both results. The turn completes. - Turn 2 in the same session: a replay of a turn 1 id returns the cached result and does not dispatch again. The same id with different arguments is refused. `admit` refuses the old id. - In five runs each call arrived 6.9 ms to 13.2 ms after its announcement. No call waited. - **Qualification** (`runner-tool-bridge.claude-tool-use.test.ts`). The pinned Claude Code client sends `_meta["claudecode/toolUseId"]` on each `tools/call`, with the id that the API issued. This test must pass before a Claude Code version change. - **Admission unit tests** (19): announced then called, called then announced, never announced, turn end during the wait, cancel, an id from the previous turn, reuse, two concurrent calls with one id, two turns with live calls, a subagent call, a bad id. - **Sidecar process tests** (2), with a fake host, for Claude and for Codex. The Codex run shows no change. - **Bridge unit tests** (5). Mutation checks: - Change the announcement predicate so that it matches nothing: the real ACP test and one unit test fail. - Remove the bridge option in the real ACP test: the test fails. - Count only `tool_call_update`: the unit test "counts only an opening ACP tool_call event as an announcement" fails. Live run with a real Claude account, not in CI: the real ACP runtime (ACPX 0.22, the patched `claude-agent-acp`, the pinned Claude Code client), a real tool bridge, and the real Anthropic API with model `claude-sonnet-5-5`, on the code at the top of this stack. In three sessions, 8 Paperclip tool calls were admitted and none was refused. This includes calls in a prompt turn, in a cycle that Claude started itself, and in a later turn of the same process. Each call arrived 0.8 s to 1.7 s after its announcement, so no call waited for a late announcement. Not run: a turn through the real sidecar process with the real host. That needs a qualified provider installation, the runtime sandbox and the provider lifetime lease. Not run: the Linux builds of the pinned client and a test under load. Not run: `pnpm verify` and the full suite. CI supplies that coverage. ## Risks - **This changes each Paperclip tool call that Claude makes.** If the announcement does not reach the turn event stream in production, the sidecar refuses the call after 5 seconds. The real ACP test covers the stream of ACPX 0.22 with the pinned provider. It does not cover the real sidecar host. - **The 5 second limit is not tested under load.** A slow host can delay the announcement. The call is then refused, and Claude sees a tool error. - **A Claude Code version that stops sending the id breaks all Claude tool calls.** The qualification test finds this before the version changes. - **The admitted-id ledger is in sidecar memory.** Today the sidecar restarts at each turn boundary, so the ledger is small. The limit of 8,192 ids refuses new calls when it is full. - **Subagent calls and calls from a cycle that Claude starts itself** are covered by unit tests and fake-host tests, not by the real ACP stream. - **An admitted call that is in flight when its turn ends** is refused by the existing turn cleanup. Claude Code can leave the HTTP request open. The bridge stops it only on its own cancel or close, as before. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the change, reviewed the diff, asked for the real ACP runtime test, and ran the verification and a mutation check again. - Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with tool use (shell, file edits, local test runs), wrote the change and the tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
89f486cdab |
fix(heartbeat): run one dispatch pass per agent, back off refused claims, and enforce the run limit in the claim (#21433)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat queue starts an agent's queued runs. Each trigger (a run finishes, a wake arrives, a sweep runs) asks for one dispatch pass of that agent > - An agent could run more runs than its `maxConcurrentRuns`, dispatch passes could stack up without a limit, and every pass offered the same unclaimable runs again > - On hosted staging ( |
||
|
|
bc4bc61f9c |
fix(ui): restore chat submission on plain HTTP origins (#20308)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators send task replies and chat messages from the board UI. > - Browsers can omit `crypto.randomUUID` on a plain HTTP origin. > - The send handler calls that API before it sends the request. Its error handler restores the draft. > - The UI already has a UUID helper with a fallback for this case. > - This pull request uses that helper in the task and chat send paths and image upload state. > - Operators can send messages and retain the existing request receipt behavior on these origins. ## Linked Issues or Issue Description Fixes: #15846 Refs #15495, #16090, #15135, #17081. Related open proposals: #15937, #16522, #16947. They remain unmerged. This change adds independent regression and browser verification. It does not establish a cause for every intermittent send failure in those reports. ## What Changed - Use `createUuid` for both composer attempt IDs, both request ID fallbacks, and image upload state. - Test five successive sends in task and chat modes without `randomUUID`. - Test upload receipts, retry request identity, the next draft, and first-message creation without that API. ## Verification - Base: `d10c6e0e8b3005cb49d0732a670ab1a851833b01`. - Six focused regression cases fail with the original calls. The same cases pass with this change. - The three affected test files pass: 506 tests. - `pnpm --filter @paperclipai/ui typecheck`, `pnpm --filter @paperclipai/ui build`, and `pnpm check:token-gates` pass. - `pnpm verify` ran 460 UI files: 7,895 tests pass and one is skipped. One CLI startup suite failed because this environment supplied empty AI endpoint variables. That suite passes when those variables are unset. The aggregate run also reported changed inputs, so it is not a reusable verification pass. - Chromium, real local UI and API, empty synthetic instance: the original Send action sends no request and retains the draft when `randomUUID` is absent. With the fix, five task replies, a scheduled-task reply, a Classic reply, and five standing-chat messages save. Request IDs are distinct and drafts clear. An image uploads and its receipt is attached to the saved comment. - Browser scope: loopback HTTP with `randomUUID` removed before page code runs. This models the missing browser API. Service workers are blocked for the fixed request-count checks. Telemetry is disabled. No agent runs start. This does not test a remote LAN browser, provider execution, or intermittent failures after real agent exchanges. - Required CI and Greptile review are pending at PR creation. ## Risks - Low risk. The change uses the existing UUID helper and keeps the send and receipt contracts. - Other causes of intermittent send failures may remain. This change only addresses the reproduced missing-UUID failure. ## Model Used - OpenAI GPT-6 (`gpt-6`), with reasoning, tool use, and code execution. The harness does not disclose a more specific revision or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
d738ed5ee4 |
fix(ui): show each related task once in the Tasks tab (#20315)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Tasks tab shows related tasks in separate sections. > - A task can be both a subtask and a task created from the current task. > - The tab showed that task in Subtasks and again in its project group. > - This pull request removes tasks from project groups when another section already shows them. > - Users can scan each task once and keep the subtask progress view. ## Linked Issues or Issue Description **What happened?** The Tasks tab shows the same task in Next action or Other subtasks and again in a project group. **Expected behavior** Show each task once. Hide a project group when all its tasks appear in other sections. **Steps to reproduce** 1. Open a task that created subtasks in the same project. 2. Open the Tasks tab. 3. Compare Next action and Other subtasks with the project section. **Paperclip version or commit** Reproduced in component tests on source commit `189fefb99183`. **Deployment mode** Built from source. The fault is in the shared UI component. No related open pull request was found in searches for duplicate tasks and TaskDetailTasksPanel. ## What Changed - Exclude task IDs already shown as subtasks, ancestors, the creation source, or a root blocker from project groups. - Show a creation source that is also an ancestor only in Created from. - Hide empty project groups, including No project. - Use an index for project lookup. - Update the product contract and regression tests. ## Verification - Before the fix, four regression cases failed because task rows or links appeared twice. - After the fix, both relation-panel test files pass: 19 tests. Run from `ui`: `node ../node_modules/vitest/vitest.mjs run --config vitest.config.ts src/components/task-detail/TaskDetailTasksPanel.test.tsx src/components/task-detail/TaskDetailRelationsPanel.test.tsx`. - Tests cover active and completed subtasks, progress, independent collapse, duplicate input rows, empty project groups, ancestors, the creation source, and a root blocker. - A test with 800 tasks and 120 projects checks unique rows and bounds task and project lookups. - `pnpm check:token-gates` and `git diff --check` pass. - The first CI typecheck found an incomplete blocker-attention test fixture. The fixture now supplies all required fields. The 19 focused tests pass again. - `pnpm --filter @paperclipai/ui typecheck` passes after `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps`. The first local attempt lacked the built SDK; the retry built that dependency. - `pnpm verify` returns exit 2 because the product document requires broad CI coverage. CI typecheck, build, general tests, browser shards, and serialized server suites pass on the current head. - Full local preflight could not finish: the OpenCode CLI install hook failed, and Cargo is absent. The UI tests ran after installing dependencies with install hooks disabled. - No full-stack browser acceptance was run. This change has component DOM evidence. - Greptile gives the current head `f3bfd8fb47154810b85578ee6731246238caf67a` 5/5 with no review threads. - All 58 current-head checks pass. [CI attempt 2](https://github.com/paperclipai/paperclip/actions/runs/42657701260/attempts/2) succeeded on `f3bfd8fb47154810b85578ee6731246238caf67a`. - The first Rust CI job failed in `reaping_stops_a_dead_runners_provider_and_its_escaped_tool_job`. Its report listed one unattributed orphan after cleanup. The same Rust code passed in the previous run. [One job retry](https://github.com/paperclipai/paperclip/actions/runs/42657701260/job/129311740923) passed without code changes. The original failure is retained here; its exact cause was not established. This PR does not change Rust code. ## Risks - Low risk. This changes UI grouping only. Stored task relationships and API data stay the same. - A project section can disappear when every task in it already appears above. This is the expected result. ## Model Used - OpenAI GPT-6, through Codex, with reasoning, repository tools, and code execution. The runtime does not expose a more specific model build or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
c6301c92ff |
fix(ci): keep the typecheck job working on stacked PRs and stop its CLI guidance test timing out (#21451)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the required `Typecheck + Release Registry` job from the trusted workflow on `master` > - #21404 replaced that job's six sequential steps with one script, `scripts/ci-typecheck-lane.mjs`, that runs independent checks in parallel > - PR runs take the workflow from `master` but check out the PR's merge commit. A stacked PR's merge commit is built on its base branch, so it can lack a script that only `master` has > - The lane also ran the CLI guidance test beside the typechecks at a lower CPU priority, and one of its cases is slow enough to cross its 15 second timeout there > - This pull request makes the job fall back to the earlier commands when the script is absent, and runs the CLI guidance test before the parallel checks > - The benefit is that stacked PRs typecheck again and the job stops failing on a timeout that has nothing to do with the PR ## Linked Issues or Issue Description Refs #21404 **What happened?** In the first eight PR runs after #21404 merged (2026-11-13 18:06Z), the typecheck job failed three times for reasons unrelated to the PRs: - Twice on #20617, a PR stacked on `feat/ssh-agent-ai-accounts`: `Error: Cannot find module '/home/runner/_work/paperclip/paperclip/scripts/ci-typecheck-lane.mjs'` (jobs 132076887259 and 132079570039). The job failed 0 seconds into the step. - Once on run 43480469363 (job 132075678943): `CLI guidance safety failed with exit code 1`, from `Error: Test timed out in 15000ms.` in `allows only exact-allowlist pnpm paperclipai commands on every guidance surface`, which took 17059 ms. The same case passes in about 5 seconds when the test runs alone. **Expected behavior** The typecheck job passes or fails on the PR's own code, including for stacked PRs. **Actual behavior** Stacked PRs always fail the job. Other PRs fail it intermittently (1 of 6 runs that reached the lane). **Steps to reproduce** Open a PR whose base branch does not contain `scripts/ci-typecheck-lane.mjs`, or run the lane on a loaded 4 vCPU runner. ## What Changed - `.github/workflows/pr-trusted.yml`: the lane step runs only when `scripts/ci-typecheck-lane.mjs` exists in the checkout. A new step runs the six earlier commands, in their earlier order, when it does not. Only the `typecheck_release_registry` job changed. - `scripts/ci-typecheck-lane.mjs`: the CLI guidance test moved from the parallel checks to the end of the sequential preparation steps, at normal priority. - Tests: `scripts/__tests__/ci-typecheck-lane.test.mjs` and `.github/scripts/tests/ci-plan.test.mjs` assert the two conditions, that the earlier commands appear only in the fallback step, and that the CLI guidance test is a preparation step. - `doc/DEVELOPING.md`: the "Typecheck lane in CI" section describes both. ## Verification - `node --test scripts/__tests__/ci-typecheck-lane.test.mjs`: 8 pass. - `node --test '.github/scripts/tests/*.test.mjs'` (the `policy` job's command): 807 pass. - `node --test scripts/__tests__/runner-release-crate-cache.test.mjs`: 11 pass. - Not exercised by this PR's own CI: PR runs use the workflow from `master`, so the new conditions first run after merge. The script change does run here, because the script is read from the PR's checkout. - Not measured: the lane's wall time with the CLI guidance test moved. The test takes 2 to 5 seconds alone, so the expected cost is that much. ## Risks - The fallback step repeats six commands in the workflow. It is needed only until open stacked PRs are based on a branch that has the script, and can be removed later. - `hashFiles` returns an empty string when no file matches, which is what the two conditions rely on. - If a future PR deletes or renames the script, the job silently uses the slower commands instead of failing. The wiring tests would fail in that PR. ## Model Used Claude Fable 5.2 (`claude-fable-5-2`), via Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
aa4a06653c |
fix(native): name the recorded turn failure when a native run fails (#21235)
Failed native runs now report the runner's recorded turn-failure code instead of the generic unclassified fallback, and the Sentry run-failure fingerprint includes runner:<code>. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
38821e061f |
perf(runner): serialise only changed journal records for a checkpoint (#21437)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - One server process controls every running agent, and its main thread limits how many agents can run at the same time > - Each agent has a controller journal. The controller saves a checkpoint of this journal after each control event and before each frame it sends > - Each checkpoint converted every command and every event of the journal to JSON again, also when only one record changed. The cost grew with the journal length (up to 500 commands and 4,096 events) > - This pull request keeps the JSON of each record and uses it again while the record does not change > - The benefit is that a checkpoint converts only the records that changed, so the main thread does less work for each agent ## Linked Issues or Issue Description Refs #21411 (this change sits on top of its checkpoint queue). **What existing behavior does this improve?** The controller journal checkpoint in `packages/paperclip-runner/src/control-plane/durable-prp-control-plane.ts`. Each flush calls `captureCheckpoint`, which runs `JSON.stringify` on every command and every recovery event of the journal. **Current behavior** A flush after one new event converts the full journal to JSON. With 4,096 events, one flush made about 5 MB of new JSON strings and used about 14 ms of CPU on the event loop. A hosted CPU profile with about 50 agents showed about 3% of the main thread in this path. **Proposed behavior** A flush converts only the records that changed since the last flush. The other records use the JSON that the controller made before. The saved records are byte-identical to the records of the full conversion. **Reason and benefit** The target is 1,000 agents in one controller process. Checkpoint CPU must not grow with journal length. After this change, one flush with 4,096 events makes about 2.5 KB of new JSON and uses about 2 ms of CPU. ## What Changed - Add `CheckpointRecordCache` to `durable-prp-control-plane.ts`. It keeps the JSON and the part name of each command and event record, keyed by the record object. It uses the JSON again only when the record has the same keys, in the same order, with the same values (`===`). Commands change some fields in place (`status`, `result`, `controllerSeq`), and this comparison finds these changes. - `captureCheckpoint` accepts the cache as an optional fourth argument. Only `setCheckpointPersistence` and `flushCheckpoint` pass it. All other callers (decode, legacy import, bootstrap inspection) do the full conversion as before. - `checkpointChanges` uses loops instead of array copies. Records that did not change keep the same string, so the comparison is by reference. - `queueCommand` stores a copy of the command payload (`structuredClone`). The journal must own its nested objects, because the cache compares nested objects by reference. - `durableRecoveryInternals` exposes `captureCheckpoint` and `createCheckpointRecordCache` for tests. - Regions changed: `checkpointChanges` and the new cache above `captureCheckpoint`; `captureCheckpoint`; `durableRecoveryInternals`; one field of `DurablePrpControlPlane`; the `captureCheckpoint` calls in `setCheckpointPersistence` and `flushCheckpoint`; the `payload` field in `queueCommand`. The `#event` path is not changed. - No change to what a checkpoint saves, when a flush is awaited, or the order of event commit, acknowledgement and checkpoint. The "at most one waiting checkpoint" behavior from #21411 is not changed. ## Verification - New `durable-prp-checkpoint-records.test.ts`: - A journal with 499 commands and 4,000 events gets one new command. The test counts `JSON.stringify` calls on journal records during the flush. Before: 4,499. After: 0 calls on existing records, at most 2 in total (the size check of the new command and its checkpoint record). The delta contains only `command:500`. The test then restores a new controller from the checkpoint and compares its records with a full conversion without the cache. I saw this test fail on `origin/master`. - A property-style test with 5 seeds and 600 random steps each: add, settle and remove commands (with `controllerSeq` renumbering), add, replay and evict events, streaming and usage events, `structuredClone` and spread copies of the state, header counters, the same value written again, and a key-order change. After each step, the records with the cache must be byte-identical to the records without the cache, in the same order. I checked that this test fails when the field comparison is removed (cache by object only). - Existing `durable-prp-control-plane.test.ts` passes. On this loaded machine, some 5-second timing tests time out with and without this change (they also time out on `origin/master`), and pass when I run them alone. - Benchmark: `PAPERCLIP_CHECKPOINT_FLUSH_BENCHMARK=1 pnpm exec vitest run src/control-plane/durable-prp-checkpoint.benchmark.test.ts` (from `packages/paperclip-runner`). Each case commits one new event and flushes with a no-op save, 200 times. CPU time is noisy (load average about 40), the byte and record counts are exact. | Journal | CPU per flush, before | CPU per flush, after | JSON bytes per flush, before | after | Records converted per flush, before | after | | --- | --- | --- | --- | --- | --- | --- | | 10 commands, 10 events | 207 µs | 64–80 µs | 20,975 | 2,465 | 20 | 1 | | 500 commands, 10 events | 1,874 µs | 505–511 µs | 375,068 | 2,466 | 510 | 1 | | 10 commands, 4,096 events | 14,013 µs | 1,655–2,343 µs | 5,174,786 | 2,473 | 4,106 | 1 | - `pnpm check:module-boundaries` passes. `tsc -p packages/paperclip-runner/tsconfig.json --noEmit` passes. ## Risks - The cache assumes that nested `payload`, `result` and `envelope` objects of journal records are not edited in place after they enter the journal. I checked the module: results are stored with `structuredClone`, envelopes with `structuredClone`, and events are replaced, not edited. This change also copies the payload in `queueCommand`. If a later change edits a nested object in place, the checkpoint could keep old JSON. The property test shows top-level edits are found. - Some per-record work is still proportional to the journal (a Map of part names and a field comparison for each record). It is about 0.4 µs for each record. Removing it needs dirty tracking in every state mutator, which this change does not do. - No change to the saved format, so old and new servers read the same checkpoints during a deploy overlap. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use (shell, file edits) in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
37393f5dc5 |
fix(browser-tracing): send API spans only for measured page loads, without timing events (#21441)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted instances can send browser page-load traces to Honeycomb (#21156, #21392) > - On staging, one hour in one workspace sent about 64,000 Honeycomb events, and only about 15 were `page.load` > - Two sources make the volume: nine network timing span events on each API request span, and request spans for background polling in idle tabs > - The page-load metrics use neither of them > - This pull request drops the timing events and records API request spans only while a page load is measured > - The benefit is about 30x fewer events in a browser test, with the same page-load data and the same server trace joins ## Linked Issues or Issue Description Refs #21156, Refs #21392 **What existing behavior does this improve?** Browser tracing exports a span for every same-origin `/api/` fetch and nine network timing events on each of those spans. **Subsystem affected** UI browser tracing (`ui/src/lib/browser-tracing/`), on hosted instances with browser tracing turned on. **Current behavior** Honeycomb stores each span event as its own event. Each request span becomes about 10 events. Idle tabs poll (for example live runs every 3 s on a task page), so most spans are requests that no page load made. On staging: about 6,600 `GET` spans plus about 6,400 of each timing event per hour, against about 15 `page.load` spans. **Proposed behavior** Request spans carry no timing events. A request is recorded only when it starts while a `page.load` measurement is open. Other requests send no span and no `traceparent`. **Reason and benefit** Event volume drops by about 30x for an idle tab. `page.load`, Web Vitals, and the Resource Timing backfill do not change. No exported request span lacks its `page.load` parent, and no server span points at a browser span that was never sent. ## What Changed - `sdk.ts`: the fetch instrumentation runs with `ignoreNetworkEvents: true` (an option of the installed `@opentelemetry/instrumentation-fetch` 0.221.0). - `sdk.ts`: new `measuredRequestsOnly` wraps the tracer provider that only the fetch instrumentation uses. When no `page.load` is open, it returns a non-recording span with an invalid context. Nothing is exported. The W3C propagator sends no `traceparent` for an invalid context, so the server traces that request alone, as for an unsampled tab. The `page.load`, Web Vitals, and backfill spans use the normal tracer and do not change. - Failed idle requests are not kept. The server's own request span records server errors. To keep them, the browser must record every idle request and send its `traceparent` before it knows the status, and that brings back the dangling joins. - `span-sanitizer.ts`: comment update only. - `doc/browser-performance-tracing.md`: new "Live API spans" section with the rule, the note that timing events are not sent, and updated volume and trace-header text. ## Verification - Red then green (`ui/src/lib/browser-tracing/sdk.test.ts`): - `records no span and sends no traceparent for an API request made while no page load is measured`: failed on the old code (an idle request sent `traceparent`), passes now. - `sends request spans without per-request network timing events`: failed on the old code (9 timing events on the request span), passes now. - All 56 tests in `ui/src/lib/browser-tracing/` pass. `tsc --noEmit` for `ui` passes. - `pnpm verify` exits 2 because a doc is a global CI input. CI supplies the broad coverage. - Real browser: local dev server on an isolated empty instance with 8 tasks, a stand-in OTLP collector, headless Chromium, and the tracing config injected into the session response. Scenario: hard load of the all-tasks page, click a task, then 2 minutes idle. One Honeycomb event = one span or one span event. | Name | master | this branch | | --- | ---: | ---: | | `page.load` | 2 | 2 | | `web_vital.lcp` | 1 | 1 | | `HTTP` (backfill) | 33 | 32 | | `GET` | 165 | 18 | | `POST` | 1 | 1 | | each of `fetchStart`, `domainLookupStart`, `domainLookupEnd`, `connectStart`, `connectEnd`, `requestStart`, `responseStart`, `responseEnd` | 165 | 0 | | **Total events** | **1,522** | **54** | - `secureConnectionStart` does not appear locally because the dev server is plain HTTP. On staging it is the ninth event. - Branch: the hard `page.load` has 32 `HTTP` children and the client `page.load` of the task has 19 live children (`GET`, `POST`). 0 request spans without a `page.load` parent (master: 146). 19 requests sent `traceparent`, and all 19 point at an exported span (master: 172 sent, 6 point at spans never exported). - Idle traffic in this test (210 `/api/` requests in total): `GET /api/issues/:id/live-runs` 88 times in 2 minutes, and `GET /api/health` 44 times (a dev-server-only 2 s poll). Most of the rest are one-off refetches. - Bundle (`vite build`): entry 3,826,982 B raw both, gzip 1,156,148 → 1,156,173 B (+25 B, chunk hash). Lazy SDK chunk 85,423 → 85,825 B raw, 27,723 → 27,859 B gzip. ## Risks - Low risk. The change is limited to the operator-gated browser tracing path on hosted instances. - Behavior shift: Honeycomb no longer has browser spans for requests made outside a page load, such as polling and mutations in a loaded page. Their server spans are still recorded as their own traces. - A request that starts during a page load and finishes after it is still recorded, as a child of that load. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, headless Chromium). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
08341ea134 |
fix(queue): let board users discard agent-authored queued messages (#21458)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The task composer holds messages that an agent has not consumed yet. > - A board operator needs to remove an agent-authored pending message before it starts more work. > - The queue currently grants discard only to the human author, so these messages have a disabled trash button and the API rejects the board user. > - This change derives the board discard permission in the queue snapshot and uses that same permission under the existing issue lock. > - The route checks task access before it changes the queue. The benefit is a working discard action with the existing company, privacy, revision, and dispatch guards. ## Linked Issues or Issue Description Fixes #15464. Supersedes #15465 with an independent implementation and red/green regression evidence. Thanks to @MMTE for identifying the gap and proposing a fix. Related: #18958 addresses deletion of comments that have already been delivered; this change covers pending queue entries. ## What Changed - Queue responses grant board users discard permission for agent-authored messages. Editing and other human authors' messages retain their existing permissions. - The discard use case honors the actor-specific permission rebuilt under the queue lock. Author agents keep their existing cancellation path. - The board discard route checks private-task access before deletion. The denial for a non-author agent now points to the board action instead of saying that no board action exists. - Tests cover queue states, cancellation of an empty continuation, persistence, queue linkage, audit attribution, private-task denial, and cross-company denial. The API reference describes the supported action. ## Verification - Red before the fix: the two snapshot regressions returned `canDiscard: false`; the discard regression threw `QueuedCommentMutationForbiddenError`. The route regressions also caught the disabled permission, missing private-task guard, and stale agent denial text. - Green after the fix: 30 queue snapshot and use-case tests; all five focused route regressions with real disposable Postgres; six Postgres adapter tests; 42 existing queue UI tests. - Server typecheck: `cd server && node scripts/typecheck-layers.mjs` passes. `node scripts/check-module-boundaries.mjs` and `git diff --check` pass. - The [complete PR workflow](https://github.com/paperclipai/paperclip/actions/runs/43484735280) passes on head `164200602828e23f01a8b209ff954821c5e14e9a`. [Greptile reports 5/5](https://github.com/paperclipai/paperclip/pull/21458#issuecomment-6861824451) on that same head with no open review threads. - Local scope: the 83 focused snapshot, use-case, route, adapter, and UI checks pass. The existing queue UI suite has 42 tests. The new route cases use real disposable Postgres and verify the resulting database state after reload. - Additional local full-suite attempt: 174 tests passed and 13 approval-related tests failed with cleanup timeouts, Postgres deadlocks, and duplicate-prefix errors after cleanup failed. The first failure, `queues request_confirmation resolution while its legacy source run is still running`, also fails on unchanged base code with the same 30-second `afterEach` and `afterAll` timeouts. This broader local run is not counted as a pass. No test timeout or assertion was weakened. CI supplies the broad passing result for the committed head. - `node scripts/local-verify.mjs --explain` returned exit 2 because the API documentation is a global CI input; all of its required CI lanes passed. The initial adapter attempt could not initialize local Postgres under the sandbox; the same six checks passed with local-process access. The initial cross-company fixture was corrected to use a distinct company prefix. ## Risks - This grants board operators a new action on agent-authored pending messages. The route must pass company membership and private-task access checks; the mutation retains its transaction, revision, and dispatch protections. - No schema or migration is required. The UI uses its existing discard control and error handling. - The staging task was inspected to confirm the cause. No staging data was deleted, and a deployed UI retest is not claimed by these local and CI checks. ## Model Used - OpenAI Codex, GPT-6, with reasoning, repository inspection, tool use, and code execution. The exact runtime model ID and context limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — the focused regression, adapter, and UI checks listed above; the additional broad local cleanup failure is documented separately - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
8b7b2a0701 |
fix: Recognize Git mount-boundary absence diagnostics in workspace checks (#21048)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. > [!WARNING] > This change crosses a trust boundary. See Risks. ## Thinking Path - Issue #20525 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Both callers recognized only the ordinary Git absence diagnostic. Preserve the earlier exact boundary matcher and all existing exit, signal, environment, and filesystem checks. - This pull request: Recognize Git mount-boundary absence diagnostics in workspace checks ## What Changed - `server/src/services/git-mount-boundary-message.ts`: Retain the shared complete-diagnostic matcher, which rejects partial and appended output. - `server/src/services/managed-repository-checkout.ts`: Retain boundary recognition without changing ordinary-diagnostic behavior or fallback guards. - `server/src/services/heartbeat/workspaces.ts`: Retain boundary recognition and ancestor metadata inspection. - `server/src/__tests__/git-mount-boundary.test.ts`: Add root mount, CRLF, and embedded-line-break cases through both callers. Confirmed six failures with boundary recognition disabled; restored implementation passes this suite, both neighboring suites, and typecheck. ## Verification - Reproducing test(s), added in this PR: `server/src/__tests__/git-mount-boundary.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - The repository's compile command passes with the fix applied. - 10 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. - It crosses a trust boundary. Check each crossing against what the issue asked for, and against who asked: - validation (server/src/services/heartbeat/workspaces.ts:860): Broadens the accepted Git absence diagnostics to include the mount-boundary message when classifying a checkout failure as an expected non-repository result. - validation (server/src/services/managed-repository-checkout.ts:265): Allows the mount-boundary diagnostic to bypass a previously thrown Git error when plainTaskDirectory and the existing exit, signal, and environment conditions hold. ## Model Used - opus (Anthropic Claude, via Claude Code), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #20525 #20525 also mentions, without asking this fix for it: - Non-Git directories in close readiness. (The report explicitly identifies merged #19553 as handling 'a different caller'; this patch does not modify close readiness.) - Initial managed-clone readiness. (The report says '#2977 concerns initial managed-clone readiness, rather than this diagnostic matcher.') - CI jobs 131047332629 and 131040975860 fail; fix the change only if it causes the failures. (This earlier-round conditional comment supplies no demonstrated matcher defect. .github/workflows/pr-trusted.yml:1801-1804 asserts aggregate shard status, while server/src/services/tool-access/catalog/health.ts:222 reports MCP health failures; neither changed here. Historical job causation cannot be) A search of paperclipai/paperclip's open and closed pull requests found no other pull request that names #20525. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=20525 base=156453fe7c67 report=513d758246cb2ac5 --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>canary/v2026.1113.0-canary.10 |
||
|
|
80f7989f4b |
fix: Wait for the SSH export pipeline to drain before importing the bundle (#20708)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16567 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: SSH process closure ended the destination stream while the progress Transform still held buffered bytes. Transfer success now requires both pipeline completion and SSH completion. - This pull request: Wait for the SSH export pipeline to drain before importing the bundle ## What Changed - `packages/adapter-utils/src/ssh.ts`: Use Node's pipeline to drain the stream chain, propagate stream errors, and release streams on failure while preserving SSH stderr errors. - `packages/adapter-utils/src/ssh-export-stream.test.ts`: Reproduced truncation and confirmed the fix passes with and without progress. Added failure cleanup cases; their final run and typecheck have not returned, and live SSH fixtures skipped because ssh is unavailable. ## Verification - Reproducing test(s), added in this PR: `packages/adapter-utils/src/ssh-export-stream.test.ts`, `packages/adapter-utils/src/ssh-export.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 10 existing test file(s) near the change failed, and fail on the base too, so they cannot show whether the change breaks them. - The full test suite passes except for the 17 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16567 #16567 also mentions, without asking this fix for it: - Add git bundle verify or a remote size check to provide a clearer diagnostic for short files. (The report calls this 'A cheap extra guard would be git bundle verify (or a size check sent by the remote script) before git fetch'; it is an optional additional safeguard. Empty or malformed output that ends normally still reaches the existing git fetch validation at packages/adapter-utils/src/ssh.) Other pull requests that name #16567: - #16613 (open): fix: prevent truncated SSH Git exports ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16567 base=5ab98b006359 report=6118c07135df7c3a --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
ed671f57bf |
fix: Display Ollama billers with their product names in cost views (#20420)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16573 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: The shared providerDisplayName map omitted ollama and ollama-cloud, causing all cost views using it to display the raw slugs. - This pull request: Display Ollama billers with their product names in cost views ## What Changed - `ui/src/lib/utils.ts`: Add Ollama and Ollama Cloud names in the shared formatter used by cost views. - `ui/src/lib/utils.provider-display-name.test.ts`: Reproduce the missing names and cover case-insensitive lookup and unchanged fallback behavior. Confirmed failure before the fix; all 20 selected tests and the workspace typecheck passed afterward. ## Verification - Reproducing test(s), added in this PR: `ui/src/lib/utils.provider-display-name.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 10 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 3. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16573 Other pull requests that name #16573: - #16609 (open): fix(ui): map Ollama and Ollama Cloud biller slugs to human-readable names - #16590 (open): fix(ui): format ollama and ollama-cloud provider names in cost views - #16574 (open): fix(ui): show Ollama and Ollama Cloud by name in cost views ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16573 base=8e5959c759bb report=f738d3d291d2282c --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
6f2ea3b4d5 |
fix: Complete run cancellation when lifecycle event append fails (#20215)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. > [!WARNING] > This change crosses a trust boundary. See Risks. ## Thinking Path - Issue #20116 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Cancellation awaited the lifecycle event append after cancelling the run and wakeup. An append failure rejected the stop before issue release, agent finalization, and queue advancement. - This pull request: Complete run cancellation when lifecycle event append fails ## What Changed - `server/src/services/heartbeat/run-control.ts`: Log cancellation event append failures and continue essential cleanup, matching lease-release terminalization. - `server/src/services/heartbeat/run-control-event-failure.test.ts`: Cover queued and running cancellation with an injected append failure and the real issue-release adapter. Confirmed regression failures before the fix; regression tests, neighboring tests, and final typecheck pass. ## Verification - Reproducing test(s), added in this PR: `server/src/services/heartbeat/run-control-event-failure.test.ts`, `server/src/__tests__/heartbeat-lock-release-on-reassignment.test.ts`, `server/src/services/heartbeat/run-control.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 9 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. - It crosses a trust boundary. Check each crossing against what the issue asked for, and against who asked: - logging (server/src/services/heartbeat/run-control.ts:1665): Logs the event-append error and run ID to the server log when cancellation event recording fails. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #20116 The reviewer found the base already handles: - A closed request socket produces a misleading PATCH 200 log that hides the failure. (server/src/middleware/logger.ts:89–91 detects aborted or unfinished responses and formats 'request aborted'; lines 150 and 157–158 log these at warn level with that message instead of the default status-based success message.) A search of paperclipai/paperclip's open and closed pull requests found no other pull request that names #20116. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=20116 base=ca34bed25797 report=e2609f9803f7cc0c --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
a6608bd735 |
fix: Preserve plugin UI UUID validation and test trailing line terminators (#20058)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #17332 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Plugin keys reached a UUID-column query, and Drizzle's nested PostgreSQL error bypassed the original error handler. The existing UUID guard fixes this; its non-multiline JavaScript regex also rejects trailing newlines. - This pull request: Preserve plugin UI UUID validation and test trailing line terminators ## What Changed - `server/src/routes/plugin-ui-static.ts`: Retain the existing guard, PostgreSQL UUID forms, and key fallback. - `server/src/__tests__/plugin-ui-static.test.ts`: Add route tests for four trailing line terminators and accepted PostgreSQL UUID forms. Confirmed the original lookup fails; all 58 focused tests and typecheck pass with the guard. ## Verification - Reproducing test(s), added in this PR: `server/src/__tests__/plugin-ui-static.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 2 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. - Revised in 1 round(s) of automated code review. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #17332 Other pull requests that name #17332: - #17433 (open): fix(plugins): serve a plugin UI bundle addressed by plugin key - #17335 (open): fix(server): serve plugin UI assets requested by plugin key ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=17332 base=ca34bed25797 report=b2e95a170c3be729 --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
f4159e25e4 |
fix: Remove the unused OpenCode remote config guard and document host staging (#20045)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. > [!WARNING] > This change crosses a trust boundary. See Risks. ## Thinking Path - Issue #17372 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: The helper's unused targetIsRemote guard contradicted both callers, which intentionally stage configuration on the host and transfer it to remote targets as xdgConfig. - This pull request: Remove the unused OpenCode remote config guard and document host staging ## What Changed - `packages/adapters/opencode-local/src/server/runtime-config.ts`: Remove the unused option and contradictory guard while preserving both callers' staging behavior. - `packages/adapters/opencode-local/src/server/runtime-config.test.ts`: Add a regression for isolated staging, generated configuration, cleanup, and operator config preservation. Confirmed failure before the fix; all 50 focused tests pass afterward. Typecheck was started but has not returned a result. - `packages/adapters/opencode-local/README.md`: Document host staging for local and remote execution, remote asset transfer, path replacement, and host cleanup. ## Verification - Reproducing test(s), added in this PR: `packages/adapters/opencode-local/src/server/runtime-config.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 9 existing test file(s) near the change passed. - The full test suite passes except for the 16 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. - It crosses a trust boundary. Check each crossing against what the issue asked for, and against who asked: - validation (packages/adapters/opencode-local/src/server/runtime-config.ts:107): Removes the targetIsRemote early-return check that prevented host configuration staging when explicitly set, although neither production caller supplies it. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #17372 #17372 also mentions, without asking this fix for it: - The remote per-run directory is never removed. (The report explicitly says: “Related but distinct: #14527 covers the remote per-run directory never being removed.”) A search of paperclipai/paperclip's open and closed pull requests found no other pull request that names #17372. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=17372 base=ca34bed25797 report=0010c91463516eaf --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
0eeee6ae96 |
fix: Defer chat milestone result reads and fix regression test transaction typing (#19875)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16728 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: PostgreSQL could evaluate both result_json predicates before publication filtering, detoasting already-published runs. The regression test also passed transaction handles to helpers typed for full database handles. - This pull request: Defer chat milestone result reads and fix regression test transaction typing ## What Changed - `server/src/services/chat-run-publications.ts`: Retain publication-dependent guards around both result JSON predicates in the shared milestone query. - `server/src/__tests__/chat-run-milestone-toast.test.ts`: Use the neighboring tests' transaction cast convention. Confirmed both cases fail without the guards and pass with them; all seven selected milestone test files and typecheck passed. ## Verification - Reproducing test(s), added in this PR: `server/src/__tests__/chat-run-milestone-candidates.test.ts`, `server/src/__tests__/chat-run-milestone-toast.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - The repository's compile command passes with the fix applied. - 8 existing test file(s) near the change failed, and fail on the base too, so they cannot show whether the change breaks them. - The full test suite passes except for the 15 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. ## Model Used - opus (Anthropic Claude, via Claude Code), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16728 The reviewer found the base already handles: - Milestone projection still runs on a one-second cadence. (server/src/services/chat-run-milestone-work.ts:12 describes and implements discovery on startup/committed changes with retries for failures or unfinished batches; server/src/app.ts:1269 registers this work owner. This code exists independently of the proposed predicate change.) Other pull requests that name #16728: - #18686 (open): perf(runs): read result_json once per run in run lists and index agent runs by created_at ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16728 base=d1ee25d0f722 report=5924b3e8dd6d2fe4 --> --------- Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
633c8076c0 |
fix: Allow initial plugin configuration saves and scope completed saves to their original company and plugin (#19847)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. > [!WARNING] > This change crosses a trust boundary. See Risks. ## Thinking Path - Issue #17637 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Dirty tracking did not distinguish missing configuration records from saved defaults. The earlier attempt also updated the cache using the current selection rather than the identifiers captured when saving. - This pull request: Allow initial plugin configuration saves and scope completed saves to their original company and plugin ## What Changed - `ui/src/pages/PluginSettings.tsx`: Preserve initial saves after successful loading, validation, and pending guards. Capture pluginId and companyId in mutation variables and use them for persistence, cache updates, and invalidation. - `ui/src/pages/PluginSettings.test.tsx`: Retain coverage for initial defaults, saved values, loading failures, and validation. - `ui/src/pages/PluginSettings.save-refetch.test.tsx`: Retain coverage preventing duplicate unchanged saves during refetch. - `ui/src/pages/PluginSettings.save-scope.test.tsx`: Add company and plugin selection coverage during pending saves. The company case failed before the revision; all 17 relevant tests and the sandbox typecheck now pass. ## Verification - Reproducing test(s), added in this PR: `ui/src/pages/PluginSettings.save-scope.test.tsx`, `ui/src/pages/PluginSettings.test.tsx`, `ui/src/pages/PluginSettings.save-refetch.test.tsx` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - The full test suite passes except for the 15 file(s) that already fail on the base commit. - Revised in 1 round(s) of automated code review; 1 reviewer claim(s) were first shown by a failing test. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 3. - The scope reviewer found it also changes, beyond the report: Config query is loading, errored, or disabled (no company selected) - The scope reviewer found it also changes, beyond the report: saveMutation.mutate call signature (internal) - It crosses a trust boundary. Check each crossing against what the issue asked for, and against who asked: - validation (ui/src/pages/PluginSettings.tsx:1131): The Save button's disable check is loosened from '!isDirty' to '!isDirty && !needsInitialSave', so an unchanged form can now be submitted when the company has no saved plugin_config row. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #17637 #17637 also mentions, without asking this fix for it: - Proactive company set / job-scope hole (#12109, #12655) (Report: "This is not a request to change proactive scope. That discussion belongs on #12109 and #12655.") Other pull requests that name #17637: - #17654 (open): fix(ui): allow saving default plugin configuration ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=17637 base=d1ee25d0f722 report=44fc7d94342a1efd --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
47ce5459c9 |
fix: Keep tablet task composers above the keyboard and release sizing during pinch zoom (#19725)
> [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. > [!WARNING] > This change crosses a trust boundary. See Risks. ## Thinking Path - Issue #17866 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Viewport observation and conversation sizing were tied to the mobile navigation breakpoint. The earlier attempt also retained keyboard state during zoom and excluded classic task threads from keyboard sizing. - This pull request: Keep tablet task composers above the keyboard and release sizing during pinch zoom ## What Changed - `ui/src/components/Layout.tsx`: Retain viewport observation independent of navigation width. - `ui/src/components/Layout.production.tsx`: Apply the same viewport observation to the production layout. - `ui/src/hooks/useMobileVisualViewport.ts`: Clear keyboard state during pinch zoom and handle Safari document panning for classic conversation shells. - `ui/src/pages/IssueDetail.tsx`: Mark desktop-width classic chat pages for keyboard sizing while preserving phone layouts and other tabs. - `ui/src/index.css`: Constrain classic conversation shells during keyboard display, retain main-content scrolling, and keep their sticky composer inside the visible area. - `ui/src/hooks/useMobileVisualViewport.test.tsx`: Reproduce stale keyboard state after zoom and verify recovery. - `ui/src/components/Layout.viewport.test.tsx`: Cover both layouts, both conversation shells, zoom recovery, and document pan correction. - `ui/src/pages/IssueDetail.test.tsx`: Verify classic sizing eligibility at desktop and phone widths. Regression tests, existing thread tests, and typecheck passed. ## Verification - Reproducing test(s), added in this PR: `ui/src/hooks/useMobileVisualViewport.test.tsx`, `ui/src/pages/IssueDetail.test.tsx`, `ui/src/components/Layout.viewport.test.tsx`, `ui/src/components/Layout.test.tsx` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - 10 existing test file(s) near the change passed. - The full test suite passes except for the 15 file(s) that already fail on the base commit. - Revised in 1 round(s) of automated code review. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 3. - The scope reviewer found it also changes, beyond the report: Enabled hook callers entering zoom after keyboard detection: both Layout variants, hook test hosts, and task-chat-mobile-follow Storybook - The scope reviewer found it also changes, beyond the report: Missing visualViewport or shell; disabled hook; zero, negative, non-finite, or missing viewport height; missing or malformed scale - It crosses a trust boundary. Check each crossing against what the issue asked for, and against who asked: - validation (ui/src/components/Layout.tsx:155): Passing true removes the mobile-width eligibility check for viewport keyboard handling. - validation (ui/src/components/Layout.production.tsx:182): Passing true removes the mobile-width eligibility check in the production layout. - validation (ui/src/hooks/useMobileVisualViewport.ts:12, not asked for by the report): The scroll-reset guard now accepts classic task shells as well as task-chat shells. ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #17866 A search of paperclipai/paperclip's open and closed pull requests found no other pull request that names #17866. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=17866 base=f8cb4957dcd9 report=8607f5fafe77c2be --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com> |
||
|
|
5859e7cfa6 |
perf(apps): avoid repeated writes during permission saves (#21033)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Apps give agents access to tools through saved profiles and approval rules. > - A save replaces the full set of action entries and access bindings. > - It also updates each Ask first rule and each retired rule in a separate query. > - Adding one agent can therefore cause hundreds of database round trips. > - This pull request keeps unchanged rows and batches the required changes. > - The benefit is faster saves with the same action permissions. ## Linked Issues or Issue Description **What happened?** An app permission save repeats database work for actions and agents that did not change. With 100 actions and 500 retired rules, one agent addition makes 636 queries. Of these, 599 update approval policies. Related boundary coverage: Refs #19973. This change keeps that issue's Off and agent-selection protections. **Expected behavior** Adding an agent applies the saved Allowed, Ask first, and Off choices. The save changes only the rows that need a change. Independent approval rules remain in force. **Steps to reproduce** 1. Create an app with 100 active actions and 99 Ask first rules. 2. Add 500 disabled app approval rules. 3. Save access for three agents. Then add a fourth agent. 4. Count database queries during the save. **Paperclip version or commit** Baseline: `51d77648bc62`. **Deployment mode** Built from source. The reproduction uses disposable local PostgreSQL. ## What Changed - Keep action entries and access bindings when their saved values match. - Insert new bindings in one query. Remove only bindings and entries that no longer apply. - Read managed approval rules for the selected app. Skip unchanged rules and rules that are already disabled. - Batch new or changed Ask first rules and disable stale enabled rules in one query. - Keep existing policy conditions, priority, author, and creation time when a rule changes. - Add real PostgreSQL regression tests and a repeatable benchmark. Document independent approval rules. ## Verification - Red: the original regression made 2,995 policy writes across five saves and failed the zero-write check. - Green: 636 focused service and runtime tests passed after the rebase. CI supplies the broad checks required by `pnpm verify` (exit 2). - Benchmark: `node --import ./server/node_modules/tsx/dist/loader.mjs scripts/benchmark-app-permission-save.ts --baseline-ref=51d77648bc62 --output=/tmp/app-permission-save.json`. - Local benchmark: two warmups and six samples in alternating order. Restore three agents outside each timed sample, then add the fourth agent. Median save time fell from 458.51 ms to 38.05 ms (12.0 times faster). Query count fell from 636 to 32. Unchanged policy writes fell from 599 to zero. - Compare all saved profile, entry, binding, and policy values after mapping fixture IDs and excluding generated row IDs and write times. Check Allowed and Ask first decisions for the added agent. Tests also cover Off, an unassigned agent, additive access, removal, independent policies, company boundaries, and re-enabled rules. - CI passes on `069e2c76252a19ec3c11a061de0506eccb0463d1`: [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/43162600023). This includes typecheck, build, server tests, and all eight browser shards. - [Greptile](https://github.com/paperclipai/paperclip/pull/21033#issuecomment-6815608829) gives the final head 5/5. Its one finding is resolved. The separate Paperclip Review check is still queued. Its review task stopped after a Runner failure. - Hosted request timing and the specific reported approval prompt are not verified. No live permissions changed. ## Risks - The save now retains unchanged row IDs and timestamps. Consumers must use permission values to determine access. - Existing transaction locks and permission checks remain in place. The runtime concurrency tests exercise these paths. - The benchmark measures a local service with synthetic data. It does not establish hosted endpoint latency. - No migration or startup change is required. ## Model Used OpenAI Codex, GPT-6. Exact provider model ID is not exposed in this session. Used reasoning, tool calls, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
6ad4529c5a |
fix(native): preserve pack proof across interrupted reattachment (#21246)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The server can connect again to a live Runner after a restart.
> - Each new controller claim changes provider-pack admission to
pending.
> - The claim kept the previous verified proof only in local memory.
> - If recovery failed before its callback, the next claim lost that
proof.
> - This pull request saves a small reference to the verified proof with
the claim.
> - The next claim can check the same receipt without weakening
admission checks.
## Linked Issues or Issue Description
**What happened?**
A restart recovery attempt timed out before its admission callback. The
next attempt refused the saved pack with `source_state_pending`. It then
failed with `native_provider_artifacts_admission_unverified`.
**Expected behavior**
The next attempt can reuse the previous verified proof for the same
receipt, session, Runner, and lease. Current admission must remain
pending until its normal callback succeeds.
**Steps to reproduce**
1. Save verified provider-pack admission for a run.
2. Claim that run again. Commit the pending admission state.
3. Interrupt recovery before the session admission callback.
4. Claim the same run again and reconnect to the same Runner.
5. Observe that the old code refuses the saved receipt.
Related work: #20682 and #20850 address lock waits. #21085 adds refusal
logs. #21161 addresses command waits after restart. This change
addresses lost admission proof across interrupted claims. The original
transport timeout remains a separate investigation.
## What Changed
- The admission module prepares the claim patch and its recovery source.
- The claim saves a versioned reference with the source run, receipt
digest, and previous verified state. It adds no copy of provider files
or credentials.
- A later pending claim can use that reference only if the receipt is
unchanged. Normal company, agent, session, Runner, and lease checks
still apply.
- Missing proof, unknown state, a changed receipt, or a different Runner
stays refused. The callback clears the reference when it records its
decision.
- The executor uses this helper under its existing claim lock. Existing
artifact-retention and ownership checks stay in place.
- The execution guide states the previous-release behavior. Old verified
records seed the reference. Old pending records without a reference
remain refused. Older readers ignore the field and keep their existing
refusal behavior. No migration is needed.
## Verification
- The new interrupted-claim regression failed before the fix: admission
returned false instead of true.
- Focused admission, PostgreSQL fencing, and executor lease tests passed
after the rebase: 61 tests on commit
|
||
|
|
9d9ee92d15 |
refactor(evals): remove the orphaned chat TestOps dispatch guard (#21241)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog eval harness sends selected cases to TestOps plans through GitHub workflows. > - PR #18322 added a dispatch guard script for the `chat-feature-testops` workflow. The guard checked each dispatch event before the workflow could use credentials. > - PR #18625 retired the duplicate chat TestOps plans. That PR deleted the `chat-feature-testops` workflow, the workflow step that ran the guard, and the only import of the guard function. > - The guard file stayed in the repository after that removal, but nothing calls it now. > - This pull request deletes the unused guard file. > - The benefit is less test support code to maintain. No protection is lost, because the workflow it guarded no longer exists. ## Linked Issues or Issue Description No public issue exists for this cleanup. The problem is described here, following the enhancement template. **What existing behavior does this improve?** The chat TestOps eval support under `tests/runner-e2e`. The guard script `testops-chat-dispatch.ts` verified workflow dispatch events for a retired workflow. **Current behavior** `tests/runner-e2e/testops-chat-dispatch.ts` holds a guard for the `chat-feature-testops` workflow. That workflow was deleted in #18625. A search of all tracked file content finds no import, no run command, and no test that uses the guard file today. **Proposed behavior** The repository does not keep the unused guard file. The runner-e2e typecheck program and the test tree load one support file less. **Reason and benefit** The guard protected a workflow dispatch path that no longer exists. Keeping the file suggests a live guard where none runs. Deleting it removes 31 lines of test support code. No regression coverage is lost: no test executes the file, and the contract it guarded is retired. **Breaking changes** None. The deleted file has no callers. No workflow, script, test, or documentation references it. ## What Changed - Deleted `tests/runner-e2e/testops-chat-dispatch.ts` (31 lines of test support code). No production code changed. ## Verification - Searched all tracked file content for `testops-chat-dispatch`, `validateChatTestOpsDispatch`, and `ChatTestOpsDispatch`. Only the deleted file matched. - Checked every selection path that could run the file: it is not a `*.test.ts` file; the runner-e2e harness selects test files through the catalog manifest, which does not name it; the root vitest projects do not include `tests/`; the `node --test` script lists do not name it; no workflow runs it. - Confirmed the history: #18322 added the file; #18625 deleted the workflow step `node tests/runner-e2e/testops-chat-dispatch.ts` and removed the only import and the assertions that referenced it; no commit touched the file after that. - Local limitation: this workspace has no installed dependencies, so the layered typecheck and the vitest suites could not run locally. The CI lanes on this pull request must confirm them. ## Risks - Low risk. The file had no callers. The remaining risk is a hidden reference that a text search cannot find. The typecheck lane and the CI test lanes on this pull request would find such a reference. ## Model Used - GLM by Z.ai, model id `z-ai/glm-5.3`, run through opencode by the Paperclip Runner. Extended reasoning enabled, with shell and file tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge |
||
|
|
b0447f3c76 |
fix: restrict chat task sponsors to self or admins (#21218)
## Thinking Path > - Paperclip manages AI agents and their authority to do work. > - External chat channels can create tasks for those agents. > - A task sponsor is the person responsible for that work. > - GitHub bot settings accepted another member as a sponsor from any connection manager. > - Connection management must not grant authority to act as another person. > - This change checks each sponsor on the server and limits the picker choices. > - The benefit is that ordinary members cannot assign work to another person's identity. ## Linked Issues or Issue Description **What happened?** A connection manager could save another member as a GitHub guest sponsor or automatic-task responsible member. The server checked membership but did not check authority to select that person. **Expected behavior** An ordinary manager can select only themselves. A company owner, company admin, or instance admin can select another active non-viewer member of the same company. The picker shows member avatars. **Steps to reproduce** 1. Sign in as an ordinary member with connection-management permission. 2. Open a GitHub bot's Access settings. 3. Set another member as a guest sponsor or responsible member. 4. Save. The old server accepts the request. This change returns 403. **Paperclip version or commit** Base: `9a9cd20d91edc93dba4e2295e42f0f1f9fcd4862`. **Deployment mode** Built from source. The checks use local PostgreSQL and an assembled authenticated API server. Related work: #17737 adds delegated connection configuration. #16163 separates author selection from guest access. This fix preserves both contracts. A search found no duplicate sponsor-permission fix. ## What Changed - Check every submitted sponsor in the configuration transaction. Lock the actor's membership and any required admin role through the save. - Require self-selection unless the actor has a current admin role. Check all target members in the same company. - Keep delegated connection checks, authority notifications, tool updates, and audit attribution. - Use avatar pickers for guest sponsorship and automatic-task responsibility. Default new guests to the signed-in member. - Test service and HTTP denials, admin selection, role removal, invalid targets, saved settings, and picker choices. - Test forged sponsor changes with real agent API keys and run credentials. Update the existing access-map policy. ## Verification - 67 sponsor, HTTP, picker, and settings tests pass. The picker test includes 600 members and bounds member lookups. - 93 existing GitHub management, event, policy, and tool tests pass. - The assembled connection-management fixture passes. Self-save succeeds. Both credential types receive 403 for forged guest and responsible-member sponsors. The saved revision stays unchanged. - UI typecheck and production build pass. Token gates, map checks, query-error policy, migration checks, and whitespace checks pass. - Two broad chat-test shards pass: 720 cases. The local full-suite run was then stopped; the complete local suite has no final verdict. Current-head CI remains the full verification path. - Full local typecheck and build reach Rust but cannot finish because Cargo is absent. - CI found the explicit retry override in the access query. Removed it; all 45 affected UI tests pass and the CI policy gate passed on the next commit. - Historical head `4146c89f9345efd38a69739d17e3194a7cf91980` received Greptile 5/5 and passed full CI: https://github.com/paperclipai/paperclip/actions/runs/43330028028 . Those results apply to that earlier head. - Current head `890df0322f382bb8ccc0fd3d59b25ba1e0299c58` merges master without conflicts. All eight changed files match the earlier reviewed fix. The 67 focused sponsor, HTTP, picker, and settings tests pass again. The production-bearer fixture also passes for both agent-key and run-token requests. UI typecheck, token gates, map checks, query-error policy, and whitespace checks pass. - Greptile gives current head `890df0322f382bb8ccc0fd3d59b25ba1e0299c58` 5/5 with no open review threads. All checks for this head are complete with no failures. The PR workflow passed: https://github.com/paperclipai/paperclip/actions/runs/43363400247 . This includes build, typecheck, general tests, serialized server suites, Runner checks, policy checks, and all eight E2E shards. The Snyk status also passed. - `pnpm verify --explain` returns exit 2 because the access-map change selects broad CI. The focused local checks pass. Broad checks are assigned to the CI run above. - The first broad checks inspected upstream migrations before the merge commit existed and stopped at migration classification. After the merge commit, the migration checks pass with zero new migrations. - Earlier saved-session broad checks stopped because Cargo was absent or memory was exhausted. These gaps are retained here. No live external-provider delivery or staging deployment is claimed. The configuration service owns sponsor authorization because both board and delegated API requests use it. No provider protocol or execution contract changes. ## Risks - An ordinary manager cannot save a configuration that names another sponsor, even when that sponsor is unchanged. They must select themselves or have an admin save it. - Membership and admin rows stay locked through the configuration write. A concurrent role change can delay the save. - Existing configurations are not rewritten. Each new save must pass the current checks. - Generic channel and AgentMail creation derive the sponsor from the authenticated user. Their strict input schemas do not accept sponsor overrides. ## Model Used OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size are not exposed in this session. Used reasoning, repository inspection, tool calls, code execution, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this fix does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue with the bug-template field labels - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My published branch name describes the change and contains no internal task details - [x] I have run the focused tests locally and they pass - [x] I have added relevant tests - [x] I have updated the relevant access policy documentation - [x] I have considered and documented the risks - [x] All Paperclip CI gates are green for the current head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
08be8be414 |
fix(recovery): report exact retained sandbox resumes as resumed (#21259)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A controller restart must keep the same remote execution owner. > - A retained sandbox lease stores the result of its first acquisition. > - That result can say replacement after the same allocation resumes. > - The Runner then rejects a valid resume as a changed sandbox. > - This pull request reports the verified operation as resumed. > - The saved acquisition history and identity checks stay intact. ## Linked Issues or Issue Description Refs #18934. That change makes the retained sandbox ready before preparation. This change corrects the acquisition receipt after that resume. **What happened?** A remote native run resumed the exact retained sandbox after a controller restart. The lease still carried a historical replacement receipt. Runner verification rejected it with `native_remote_recovery_lease_mismatch`, stage `runner_verification`, field `replacement_sandbox`. **What did you expect to happen?** Report the verified exact-allocation operation as resumed. Preserve the original saved history. Reject a changed, absent or expired allocation. **Steps to reproduce** Save an active lease with historical replacement metadata. Resume that exact provider allocation through `acquireForRun`. Resolve its execution target. Before the fix, the target still reports replacement and Runner verification refuses it. **Deployment mode** Native Runner on a remote sandbox. The reported provider was Daytona. ## What Changed - Return a separate operation receipt after the exact retained allocation passes the existing resume checks. - Pass the receipt through heartbeat preparation and apply it only to the execution target. - Keep the saved lease metadata and workspace unchanged. - Test historical created, replacement and resumed receipts. Test ordinary and remote-authority realization through the real execution target resolver. Check every saved metadata write. - Document the difference between saved acquisition history and the current resume operation. ## Verification - Head: `74d03e8ddadd1274891bea99a10455795c500557`. - Red: three historical receipt cases fail without the resume receipt. - Review regression red: realization writes resumed over the historical replacement receipt. The saved metadata loses the previous allocation and cause. - Green: all 69 orchestrator tests pass. Ordinary and remote-authority realization preserve history in every metadata write and return resumed in the target. - Green: 135 focused orchestrator and remote controller adoption cases pass. Identity and changed-allocation refusals remain effective. - Server typecheck passes, including its TypeScript build dependencies and test layers. - `git diff --check` passes. - `pnpm verify` exits 2 because the execution-semantics document needs broad CI. Required CI supplies that coverage. - Linux x64, Node 25.9.0. Provider boundaries are simulated. - The first post-merge attempt could not collect tests because the compiled plugin SDK lacked the new taskLease schema. Shared and plugin SDK builds passed, then the unchanged selection passed. - Full preflight stopped because Cargo is absent. Dependency installation completed. The later server TypeScript build and typecheck passed. - Live retained Runner reattachment is not verified. This workspace has no provider credentials. No rollout or original-task recovery by this patch is claimed. - Required current-head CI and automated review remain pending. ## Risks A wrong receipt could hide an actual replacement. The projection occurs only after the existing company, task, run, agent, workspace, provider allocation, expiry and directory checks pass. The provider must resume the exact allocation. Runner process and identity verification still runs. No replacement is acquired and no saved acquisition history is changed. The heartbeat change only carries the typed receipt between the owning orchestrator stages. Callers without a receipt keep their existing behavior. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution and tool use. The exact backend snapshot and context window are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |