mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
afe5b3418dbdf48aca5791f8fc8df12c7a75a246
2334
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
afe5b3418d |
fix(native): verify warm ownership before configuration rotation
Track the exact prior warm-owner run ID separately from mutable configuration. Keep configuration retirement, terminal-state checks and failed-close fences. Add current-v5 handoff regressions and document the recovery boundary. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
d5b1975621 |
fix(native): serialize warm idle expiry with turn admission (#16432)
The warm idle timer no longer closes a warm native session while a continuation has reserved or is preparing the same session scope; it re-arms instead. An idle close that has already started is registered in closingWarmNativeSessions so turn admission waits for the checkpoint, matching the other closers. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
7ec6fd0972 |
perf(ci): remove repeated test setup and fixture work (#16429)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - CI must check changes without making contributors wait for repeated setup. > - A complete CI profile showed 158 serialized Vitest launches and repeated dependency installs. > - A large fixture also transferred 40,000 files to prove two different size boundaries. > - This PR removes repeated controller, install, build, transfer, and wait work. > - It keeps isolated test processes and all useful regression boundaries. > - Paired command benchmarks check speed and test inventories before we claim a benefit. ## Linked Issues or Issue Description Refs #16423 for the complete CI profiling data path. No duplicate optimization PR was found. **Current behavior** Each serialized file starts a new Vitest controller. Each CI lane restores the pnpm store and installs dependencies. Large Git scanning and staging share one 40,000-file fixture. A readiness regression waits over a minute for the default database idle deadline. **Proposed behavior** Use one controller per serialized shard, with a fresh fork for each file. Restore exact installed dependencies and certified shared/SDK build outputs when inputs match. Keep the normal install and build paths on a miss. Separate the 32 MiB Git scan proof from staging above command argument limits. Cross a real database idle deadline with a one-second fixture setting. **Reason and benefit** The paired Linux CI benchmark reduced the complete 566-case serialized shard from 303.43s to 246.96s (18.6%). The Git/database fixture command fell from 156.54s to 33.78s (78.4%). Exact dependency/build restoration reduced setup from 56s to 19s (66.1%). These are measured command/setup times. They do not establish a controlled reduction in the full workflow's queue-to-completion time. **Breaking changes** None to application behavior. Tests still run in full. Test results are not cached. Cache misses preserve the existing lockfile resolution fallback. ## What Changed - Batch serialized files into one isolated, single-worker Vitest invocation per shard. Give each worker its own home, config and instance ID. - Add a real two-order isolation check for processes, environment, globals, timers and filesystem state. - Add exact dependency keys for all manifests, lockfile, workspace settings, patches, installer, runtime, platform, libc, pnpm, store and checkout path. Cache only installed package directories, not source fixtures. - Add a separate content key for the shared/SDK dependency graph. Retain the existing source/output fingerprint verifier and rebuild path. Only an exact canonical master push can populate normal caches. Stale lockfile fallback runs cannot publish entries. - Keep four real Git filename lanes above 32 MiB. Use a separate 4,000-file staging/deletion case above 2 MiB. Check every staged file, unusual names, cleanup and bounded commands. - Configure a real one-second PostgreSQL idle timeout in the readiness fixture. Assert that the backend connection changes and that another connection can write the parent row. - Add an opt-in, unprivileged Linux benchmark under `profile-ci`. It compares the tested merge tree with its actual first parent on one runner. It also removes installed outputs and qualifies a real cache restore using temporary PR-scoped entries. ## Verification - Local Node 24.19.0: unchanged serialized shard 255.69s; candidate 231.44s; all 566 cases pass in both. - Local fixture command: unchanged 189.13s; candidate 43.39s. Candidate scan 18.58s, staging 13.41s, database case 2.80s. All three candidate cases pass. - Negative control: moving readiness back inside the real parent-lock transaction fails the shortened regression at the parent-write assertion. The production source was restored and is absent from the diff. - Worker isolation check passes in both file orders. - 565 CI policy, partition and workflow regression checks pass with local HTTP/IPC access enabled. The first sandboxed attempt was denied listen access; it was not a code failure. - `actionlint` passes for the three changed workflows. - [All 48 main CI jobs passed](https://github.com/paperclipai/paperclip/actions/runs/38745643045) on head `5bec2edb2b194e74352ca697f3fd1d0ac03ce967`, including full typecheck, tests, build, browser E2E and release checks. All 158 serialized files and 3,250 cases passed through 9 controller invocations. Full typecheck/test/build were verified in CI; they were not all repeated locally. - [Paired Linux benchmark passed](https://github.com/paperclipai/paperclip/actions/runs/38745643127) on Node 24.21.0. Its baseline is `57fa6b358de62922cfa9a67766b2a227c434fcdf`; its candidate merge tree is `f68f1b32966090f11647502e31852130c20237cf`. The artifact contains results, exact case inventories, setup times and raw logs. Serialized case identities match exactly. All three fixture contracts execute in both variants. | Measured work | Baseline | Candidate | Reduction | |---|---:|---:|---:| | Complete serialized shard, 566 cases | 303.43s | 246.96s | 18.6% | | Git/database fixture command | 156.54s | 33.78s | 78.4% | | Dependency/build setup, cold vs exact hit | 56s | 19s | 66.1% | - [Honeycomb profile export succeeded on its second attempt](https://github.com/paperclipai/paperclip/actions/runs/38747237615/attempts/2). The first attempt failed on GitHub HTTP 503 while downloading an artifact, before export. Only the export was retried; tests were not repeated. The temporary PR-scoped qualification caches were removed after validation, reclaiming 3.34 GB. Timing artifacts remain available. - Greptile is 5/5 on the current head, with no unresolved review threads. Its P1 checkout-prefix finding was fixed in `5bec2edb2`; the corrected reporter paths and paired comparison passed. Earlier runs were superseded while this was fixed. A copy-batching experiment did not speed up the Git fixture and was replaced by the two-boundary fixture split. ## Risks - A persistent controller retains transform state longer. The local shard's maximum RSS rose from about 1.52 GB to 1.99 GB. Full CI must confirm all shards fit the runner budget. - Materialized caches are larger than metadata caches. The Linux benchmark measured real archive transfer and validation after removing installed outputs. Its hit took 19s versus 56s cold. Normal PR lanes cannot publish cache entries. Exact misses run installation. Canonical cache warming requires a frozen-lockfile install; the existing stale-lockfile fallback remains uncached. - The PR uses the trusted reusable workflow from master. The paired benchmark directly exercises the candidate dependency action; normal workflow adoption still requires this PR to merge. This PR does not change the trust references. The completed main workflow took 11m02 including queue time. No controlled full-workflow latency improvement is claimed before this workflow adopts the cache change. - Only deterministic dependency/build outputs are reused. Broad dependency-based test omission and caching successful test results are deliberately excluded from this change. - The real Runner lease minimum and cold-admission waits remain. They protect protocol boundaries and were not shortened by weakening production limits. ## Model Used OpenAI GPT-6 through Codex, with reasoning, shell execution and code tools. The runtime does not expose a more specific deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
69eb3b6eff |
perf: reduce repository admission filesystem and write overhead (#16425)
Reduce repository preparation latency by resolving each distinct workspace path once per scan, loading overlapping root provenance, and batching source retention writes. Preserve company authorization, fresh alias checks, and preparation timing. Add a reproducible benchmark and regressions for source identity retention, symlink retargeting, and bounded filesystem and database work. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5a9e6b6315 |
fix(grok): avoid false file recovery after Stop (#16427)
## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Grok Bot uses a managed Runner and a fixed external work session. > - Its admitted contract gives it no access to Paperclip workspace files. > - The Runner still has a sync descriptor to support durable recovery. > - Cancellation treated that descriptor as proof that files needed repair. > - This pull request uses the admitted access contract while preserving ownership settlement. > - Users get the correct recovery action after Stop, with native file safeguards intact. ## Linked Issues or Issue Description Refs #16219 and #16422. The duplicate search found no open Grok workspace correction. **What happened?** A real Grok Stop test cancelled its run, fenced its assignment, denied a stale progress update and cleaned up the sandbox. Paperclip also created a file-repair hold. The recovery panel told the user to recover workspace files, although the admitted external contract has workspace access set to none. **Expected behavior** Keep the stopped session fenced and require inspection before new work. Keep failed-run ownership settlement. Do not invent a file export obligation for a session that cannot access the workspace. Native sessions with writable or unknown workspace access must keep their file safeguards. **Steps to reproduce** 1. Connect a Grok Bot and assign a task. 2. Let it accept the task and save a Paperclip checkpoint document. 3. Use Stop while it is working. 4. Inspect the task recovery panel. It asks for file repair after sandbox cleanup succeeds. **Paperclip version or commit** Observed on Core |
||
|
DottaandPaperclip
|
57fa6b358d |
fix(observability): separate workspace finalization and copyback spans (#16428)
Give native workspace finalization its own span name. Keep copyback and commit as separate child measurements, with an exported-span regression that preserves the workspace operation result. Validation: 40 runtime/tracing tests and server typecheck passed locally. Final-head CI, E2E and security checks passed. Apex review: 5/5, zero findings and no unresolved threads. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
cccde7122b |
Trace execution preparation and durable response persistence (#16419)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server prepares an execution environment before it starts an agent. > - Operators need to distinguish preparation, provider wait, and response persistence. > - Existing preparation phases start separate traces, and the native session trace ends before the final response is saved. > - This pull request adds an execution-attempt trace and detailed spans at the existing work boundaries. > - The benefit is a measurable startup and settlement path without changing execution policy. ## Linked Issues or Issue Description **What existing behavior does this improve?** Operator-configured OpenTelemetry tracing for agent preparation, startup, and durable response storage. **Subsystem affected** Server orchestration, native Runner startup, and the Honeycomb run link. **Current behavior** Preparation phases start independent roots. Workspace lock waits, authorization groups, runtime asset work, provider-pack fallback, and the gap after provider completion are hard to attribute. **Proposed behavior** Use a server-owned async context for each execution attempt. Parent nested work to that attempt. Link the separately sampled native session because its ingress and queue history can begin earlier. Record transaction entry, lock waits, grouped authorization, asset preparation, artifact fallback, copyback, and response commit. **Reason and benefit** Operators can identify expensive work from recorded boundaries instead of inferring the cause from elapsed time. Nested and concurrent durations remain visible and must not be added together. **Breaking changes** No API or execution-policy change. New Observability span names and closed count attributes are documented. The existing local run-performance schema also records the additional provider-pack and harness measurements. Refs: #16178, #16205, #16330. ## What Changed - Add `task.execution_attempt` with same-run async ownership, independent root sampling, and native-session links. - Measure workspace transaction entry and settlement, advisory and row waits, source scans, reader groups, notifications, and writes. - Measure runtime selection and each concurrent instructions, skills, and MCP branch. Separate manifest, cache, materialization, and verification work. - Measure provider-pack discovery, verified reuse or upload, final verification, and remote harness state checks. - Measure workspace settlement, accounting, liveness, response authorization, comment commit, and publication enqueue. - Preserve the silent endpoint-off path, including when another library has registered a tracer. Bound repeated detail spans to 16 per name per attempt. - Extend the Honeycomb run link and document the actual coverage and remaining gaps. ## Verification - After resolving the conflict with retained-reader batching on `b98cdaea531c8447637cd4dcc19c61b3f5f41844`, all 61 targeted query, authorization, workspace-preparation and trace-boundary tests passed. The server typecheck passed. Reader tests still bound query count, avoid loading retained run payloads and enforce fresh project grants on every publication attempt. - Before that base integration, the tracing/exporter group passed 855 tests with 2 optional-SDK skips across 10 files and no inherited OTLP endpoint. It covers real PostgreSQL waits/rollback, durable response commit, assembled heartbeat preparation, concurrent parentage, sampling and tracer failure isolation. Native workspace/history tests passed 23 cases. These results remain supporting evidence for the unchanged tracing paths. - Full workspace typecheck and build passed during implementation. Full sharded CI, release validation, E2E and Apex 5/5 passed on prior head `8cebad80f3856dbcff6f3a6d5d34880ff77a20f2`. Fresh CI, security and Apex review are required on the current head before merge. - `pnpm check:token-gates` passed. No styles changed. - Earlier CI runs exposed exact-source-expression assertions changed by wrappers, missing run hashes on transaction-boundary spans, and sampling fixtures without an enabled endpoint. The fixes preserve the calls, add the declared hash and span cap, and enable tracing explicitly in the sampling fixtures. Focused repairs passed; the full prior-head CI passed after those repairs. Earlier failed run: https://github.com/paperclipai/paperclip/actions/runs/38729131366. - The full local `pnpm test:run` was stopped after the base advanced during that process. Its mixed-version results are not final-head evidence. A full local-suite pass is not claimed. - The old-head Superagent scan stalled at 35%. The current head has a fresh scan; no security pass is inferred from the old pending result. - No deployed Honeycomb acceptance is claimed by this code change. ## Risks - Enabled tracing creates more spans. Repeated detail spans are bounded, and inventories use group spans. - The attempt and native session remain linked traces. Historical queue time can precede the attempt, and nested intervals overlap. - A committed comment proves durable server storage. Publication enqueue and realtime dispatch do not prove external delivery or browser paint. - New custom attributes use only the existing run hash and validated file/byte counts. No SQL, paths, permissions, content, or exception text is added. ## Model Used OpenAI GPT-6 through Codex. The serving model ID and context window size are not exposed in this session. The agent used reasoning, tools, code editing, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7420af26fc |
fix(ci): finish large Honeycomb profile exports (#16423)
## Thinking Path > - Paperclip uses pull request CI to check changes before merge. > - The merged CI profiler records a selected full PR suite in Honeycomb. > - A real 48-job profile exposed an export failure after Honeycomb accepted only part of the trace. > - The same run exposed three fixture subprocesses that looked like incomplete test launches, one direct reporter without a launcher marker, and one overlong test name. > - The exporter needs to handle temporary ingest rejection without sending accepted spans again. > - This pull request fixes those cases and documents the trace display limit. > - The benefit is a complete, checkable CI waterfall with test-file timings. ## Linked Issues or Issue Description **What happened?** The first live `profile-ci` run on #16423 completed all 48 jobs, but the post-run exporter stopped when Honeycomb rejected part of a batch. The initial trace had no root span. Its coverage flag also reported four unfinished invocations and one invalid timing record. **Expected behavior** The exporter should pace large uploads, retry only temporarily rejected events, and publish the root after every child is accepted. Completed reporters should count as complete. Long test names should not erase test timing. **Steps to reproduce** 1. Add `profile-ci` to a CI CODEOWNER PR with the merged profiler. 2. Run the full PR suite, as in [run 38732093469](https://github.com/paperclipai/paperclip/actions/runs/38732093469). 3. Inspect the separate [failed exporter run](https://github.com/paperclipai/paperclip/actions/runs/38733393370) and the trace coverage fields. **Paperclip version or commit** `12612b06eb4332844ac4c8770a1fb6c6b7eccbc9` (PR run head, based on merged #16378). ## What Changed - Pace Honeycomb uploads and retry only transiently rejected events from each batch. - Preserve overlong test timings with bounded names and count complete reporter pairs without a launcher record. - Keep nested Vitest fixture runs from inheriting CI profiling and creating false coverage gaps. - Clarify ordinary CI access, large-trace display limits, and export behavior in the operator guide. ## Verification - [Full PR suite](https://github.com/paperclipai/paperclip/actions/runs/38732093469/attempts/1): 48 of 48 jobs succeeded; 10m 29s wall time. The selected jobs used the AWS public PR fleet. The protected exporter used a separate GitHub-hosted runner. - [Clean Honeycomb file waterfall](https://ui.honeycomb.io/paperclip/environments/test/datasets/paperclip-ci/trace?trace_id=01ed0c541b0442e613a030ffb6dc8d9b&trace_start_ts=1792065955&trace_end_ts=1792066584): 48 jobs, 822 steps, 2,622 file spans, and `ci.profile_complete=true`. It was reconstructed from that run's 45 timing artifacts after identifying three nested fixture records. The full trace also recorded 44,497 tests; Honeycomb's chart omits some deep test spans because it displays at most 32,000 spans per trace. - The original exporter stopped after a partially accepted batch. A local replay of that batch succeeded. I used the authorized local test-environment key to complete the original upload and create the file-level view. No key was added to this branch or PR. - `node --test .github/scripts/tests/ci-profile.test.mjs scripts/ci-profile/reporters.test.mjs`: 18 passed. - `pnpm exec vitest run server/src/__tests__/vitest-chat-shards.test.ts`: 1 passed. - `git diff --check`: passed. - The local `pnpm -r typecheck` and profiled fixture rerun could not start: the package-manager shim could not verify the `pnpm@9.15.4` registry signature because its fetches failed. `pnpm test:run` and `pnpm build` were not run locally; hosted CI is pending. - The updated exporter has not yet run from protected `master`; this PR remains draft until CI and review finish. ## Risks - A 48,000-span export now takes about three extra minutes due to pacing. Retry limits still fail a persistently rejected batch and keep the root unpublished. - Honeycomb may retain duplicates from an already partial export. The clean file-level evidence trace uses a separate trace ID. - The API key stays only in the protected `ci-profiling` environment. PR jobs write timing artifacts but never receive it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — see `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code execution, GitHub API, and browser inspection. The runtime does not expose the exact serving model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
ea4aa185b0 |
perf: batch retained-reader authorization reads (#16424)
Batch retained-reader project checks through the existing multi-actor read API. Share raw database reads within one call, retain actor-specific privacy and supplied membership data, and avoid loading large saved-run payloads for identity checks. Keep current-reader writes, distinct saved-run policies, company root fences and later-call revocation checks. Add query-count and access-control regressions. All 255 affected cases and full repository typecheck/build pass. Hosted response-time acceptance remains separate. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7f9d3d1929 |
fix(grok): avoid routine wakes for proactive admission (#16422)
## Thinking Path > - Paperclip lets people manage AI agents and their work. > - Grok Bot can request governed work from its own conversation. > - Managed Runner admission can take longer than the initial MCP response. > - The dispatcher treated a delayed offer as a reason to start another routine. > - The original conversation was already waiting for that same work. > - This pull request keeps admission in its original request and cancels old prepared echoes. > - Users get one task interaction while normal assigned-task wakes remain available. ## Linked Issues or Issue Description Refs #16219 and #16394. The duplicate search found no other proactive Grok correction. **What happened?** A real idle Grok conversation created one Paperclip task and reached accepted completion. During admission, the dispatcher also sent a routine webhook after the offer was ten seconds old. Two acceptance requests and two title writes were recorded. The extra routine can compete with the original conversation. **Expected behavior** Immediate and delayed Bot-initiated admission stay with the original request. The Bot checks the same request ID and arguments until admission succeeds or fails. A new routine must not echo that work back to the Bot. **Steps to reproduce** 1. Ask an idle Grok Bot to create and complete a Paperclip task. 2. Let managed admission return pending. 3. Leave its offered assignment unaccepted for more than ten seconds. 4. Run the dispatcher. It sends another routine webhook for that proactive offer. **Paperclip version or commit** Observed through the actual Grok client on Core |
||
|
DottaandPaperclip
|
250429942e |
fix(agents): report actionable lifecycle failure steps (#16418)
## Thinking Path > - Paperclip helps people manage AI agents for work. > - New agents must pass setup and execution checks before they can run. > - The lifecycle worker saved the same retry message for every failure. > - Users could not identify the failed step or correct its configuration. > - The UI also described failed work as work still in progress. > - This pull request names the failed step and gives a safe corrective action. > - Users can identify configuration problems and understand automatic retries. ## Linked Issues or Issue Description **What happened?** A new hire showed a paused status and “The lifecycle step failed. Retry the operation.” Repeated retries did not explain the failure. A successful retry cleared the original error. This change cannot recover an error that an older server already cleared. **Expected behavior** Name the failed setup step. Give a useful action for known execution test failures. Explain automatic retry. Keep provider output and credentials out of lifecycle errors. **Steps to reproduce** 1. Create a disposable agent with an unavailable model or invalid execution command. 2. Open its profile while verification fails. 3. Inspect the error, correct the configuration, and select Retry now. 4. Confirm that the failure disappears after the agent becomes ready. Related change: https://github.com/paperclipai/paperclip/pull/16155 shows pending setup progress in creation and assignment. This change improves the cause and action shown when setup fails. ## What Changed - Save the failed lifecycle phase and step in the existing lifecycle error field. - Classify authentication, model, command, directory, timeout, and response failures from execution test check codes. - Use fixed error messages. Do not retain raw provider messages, check details, or credentials. - Share the allowed failure codes with the lifecycle log. - Show failed-state labels and automatic retry guidance. Rename Retry to Retry now. - Extend worker, database, and UI regression coverage. Update the agent verification maps. ## Verification - Passed 105 focused tests across lifecycle diagnostics, database lifecycle commands, saved harness verification, and AgentLifecycleStatus. - Passed server TypeScript compilation with `pnpm --filter @paperclipai/server exec tsc --noEmit`. - Passed UI typecheck and production build. - Passed module boundary checks, token gates, character sync, map validation, and `git diff --check`. - Used Playwright to inspect the production component with a simulated model failure. Confirmed the failure text, retry guidance, retry callback, and recovered state. Captured a screenshot. This component check does not prove a live provider run. - Attempted `pnpm -r typecheck` and `pnpm build`. Both require Rust cargo, which is not installed in the local execution environment. - Started `pnpm test:run` locally. No final local suite result was recorded before handoff. The complete PR CI workflow passed. - Passed all latest-commit CI gates, including typecheck, build, general and serialized tests, Rust checks, and all eight browser test shards: https://github.com/paperclipai/paperclip/actions/runs/38726198774 - Greptile scored the latest commit 5/5 with no findings. There are no review threads to resolve. ## Risks - Failure classification depends on existing adapter check codes. Unknown codes use a safe fallback message. - A known code can describe only the checks reported by the adapter. - No database migration is required. The error field remains a string. - Retry schedules, leases, readiness, approvals, and budget controls keep their current behavior. - Full build verification requires a runner with Rust cargo. ## Model Used - OpenAI GPT-6 through Codex. Used reasoning, code editing, browser checks, and tool execution. The runtime does not expose the exact deployment ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7fc2dc8b32 |
fix: isolate workspace snapshots from controller work (#16414)
Move native sandbox snapshot captures off the controller callback queue while preserving the existing capture and restore contract. Keep legacy adapters inline, bound captures to two workers, and verify cleanup and reordered worker completion. Validation: 461 affected cases, full typecheck/build, exact-head CI/E2E/Superagent and Apex 5/5. The full local canonical suite remains running; no full local pass or under-30-second staging response is claimed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
8365872743 |
fix(codex-local): retry ACP model capacity failures as transient upstream (#16407)
Classify typed Codex ACP `service` failures about model capacity or server overload as `codex_transient_upstream` / `transient_upstream`, so the existing bounded transient retry and continuation recovery apply. In `unexpected` Sentry mode, skip only runs carrying that full classified evidence; `all` mode still reports them. |
||
|
Devin FoleyandPaperclip
|
72ae600742 |
test(server): reuse comment wakeup route modules (#16410)
## Thinking Path > - Paperclip manages AI agents and their work. > - Task routes decide which agent wakes after a comment or assignment change. > - The comment-wakeup suite reloads the large route module before each test. > - The worker exhausts its memory before all 34 cases finish. > - This PR loads the module once and resets mock behavior between cases. > - The suite keeps every assertion and runs within the existing memory limit. ## Linked Issues or Issue Description Refs #16017. Related fixes: #16396 and #16404 cover other route suites. The [failed serialized shard](https://github.com/paperclipai/paperclip/actions/runs/38706528509/job/116333628474) completes 26 of 34 cases before a 4 GiB heap OOM. It also logs `runner.select is not a function` when real external-object sync reaches the empty transaction fixture. A duplicate search found no open PR for this suite's memory failure. ## What Changed - Move module reset, mock registration, and initial route import into `beforeAll`. - Mock external-object sync at its direct service import. - Reset mock implementations between cases, then restore the fixture defaults. - Keep all 34 test cases and every assertion unchanged. ## Verification - Node 24.21.0, matching the failed CI job. - Unchanged suite with `NODE_OPTIONS=--max-old-space-size=4096`: worker receives SIGKILL after 31/34 cases in this 4 GiB container. - Fixed suite under the same heap limit: 34/34 pass in 5.06 seconds; peak child RSS is 654 MiB. - Fixed suite with a 1 GiB heap and shuffled order (seed 394): 34/34 pass. - Compared all test bodies and assertions against master; they are byte-for-byte unchanged. - `git diff --check` passes. - Workspace typecheck reaches the Rust stage, then stops because this container has no `cargo`. - Master has a stale lockfile. Local setup used `pnpm install --no-lockfile`; Vitest resolved to 5.0.4 and Supertest to 7.3.1. - Local build stops because Node.js headers are missing for the SO_PEERCRED addon. - The local full-suite command was bounded to three minutes and did not finish. - [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16410#issuecomment-6188812731), with no findings or review threads. - [CI run 38712230280, attempt 2](https://github.com/paperclipai/paperclip/actions/runs/38712230280/attempts/2) passed on this head. All nine serialized shards, general tests, typecheck, build, Rust checks, and e2e checks passed. - The first attempt hit an unrelated legacy-recovery timestamp flake. Its [single retry passed](https://github.com/paperclipai/paperclip/actions/runs/38712230280/job/116349967086). - Current check inventory: 53 successful check runs, two skipped Storybook checks, and a successful Snyk commit status. Reproduce the focused check with: ```sh NODE_OPTIONS=--max-old-space-size=4096 pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issue-update-comment-wakeup-routes.test.ts --pool=forks --isolate ``` ## Risks Low risk. Only test setup changes. Production routes, workflows, and the lockfile are unchanged. The shuffled run checks that mock behavior does not leak between tests. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. This runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandClaude Opus 5.5
|
11d69d04c7 |
fix(native-runtime): report remote recovery labels thrown before coordinator save (#16406)
Restart reattachment can fail in the executor before the native coordinator saves its result. The run failure report now reads the closed remote recovery stage and reason from the caught error when the saved result lacks them. The saved result keeps priority and the same closed-vocabulary validation applies. Add a regression for thrown, wrapped-transport, saved-priority, and out-of-vocabulary cases. Full current-head CI and Greptile 5/5 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
307c27e3d8 |
fix(native): reclaim task execution before same-run recovery
Restore the original task execution binding atomically during native same-run recovery. Preserve current owners, stops, pauses, assignments, and dependency waits. Isolate candidate failures, bound deferral writes, prioritize undated retries, and allow recovery ahead of queued waiters without premature queue promotion. Add PostgreSQL attachment, ownership, contention, fairness, and queue regressions. Verification: 99 focused tests, workspace typecheck, build, full current-head CI, and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
cd77c1c4dc |
test(server): reuse remaining serialized route modules
Load route modules once per suite while keeping every test case and assertion. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
af4191f45e |
fix: keep workspace notices from suppressing final replies (#16400)
Keep stored system notices visible without letting them occupy the final reply slot. Read presentation metadata in live and recovery selection, and cover explicit replies, progress receipts, company boundaries, and native replay. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8d9d2f9ddf |
fix(recovery): preserve deliberate stops and dependency waits
Preserve deliberate assignee blocks and operator stops during dependency and run-release recovery. Attribute native waits to their bound owner. Ignore unrelated edits and shared-workspace exports from another issue. Continue after genuine dependency changes while preserving independently authorized user input. Keep the original contributor's implementation and review history. Add native ownership, unrelated-edit, later readiness, and shared-workspace regressions. The shared-workspace loop was reproduced before the fix. Co-Authored-By: Devin Foley <devin@paperclip.ing> Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
c68a483cd9 |
fix: replay prepared warm seeds when generation caching is unavailable (#16399)
Replay this preparation’s verified private workspace seed after failed warm validation, including when the generation cache refuses publication. Preserve host-current semantics and rebuild if the private seed is missing or changed. Verified 89 focused cases, full typecheck/build, exact-head CI/E2E/security and Apex 5/5. Live PAP latency acceptance remains separate. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
5f463dd625 |
test(server): reuse route modules across test cases (#16396)
## Thinking Path > - Paperclip uses server routes to manage agents and their work. > - Route tests check permissions, login sessions, skills, and file downloads. > - Four suites reload large module graphs before every test. > - Their workers exhaust the heap and block dependency updates. > - This pull request loads each route graph once per suite. > - Each test still gets a fresh app and resets its fixture state. ## Linked Issues or Issue Description Refs #16017. Related: #12987 addresses repeated logger transports through a different production change. **What happened?** CI run 38293835699, attempt 2, aborted four server test workers. The affected suites cover attachments, device login, agent skills, and setup tokens. Three failed log tails show heap exhaustion. The separate `db.transaction` warning came from the document-annotation suite, which passed. **Expected behavior** All route cases complete within the worker heap limit and keep independent test state. **Steps to reproduce** Run the unchanged agent-skills suite from master with `NODE_OPTIONS=--max-old-space-size=1536` and `--logHeapUsage`. The worker aborted after 66 of 90 cases, at a reported heap of 1,532 MB. **Paperclip version** The fix is based on master commit `444d5854b0f53b19d138f060a267cfd10171c7b3`. The original dependency PR head is `7b74c3f248731943154547385b1de5eb6562bf01`. **Deployment mode** Local tests and CI. ## What Changed - Move module resets and mock registration from `beforeEach` to `beforeAll` in four route suites. - Load the agent-skills route modules during suite setup. - Keep each test's app, fixture resets, and login cleanup. - Remove two comments that describe the old reset behavior. ## Verification All 193 tests in the four affected suites pass with a 1.5 GiB heap limit: ```sh NODE_OPTIONS=--max-old-space-size=1536 pnpm exec vitest run --project @paperclipai/server \ server/src/__tests__/agent-skills-routes.test.ts \ server/src/__tests__/agent-device-login-routes.test.ts \ server/src/__tests__/issue-attachment-routes.test.ts \ server/src/routes/setup-token-route.test.ts --reporter=verbose --logHeapUsage ``` - Local runtime: Node 25.9.0, Vitest 5.0.4, Supertest 7.3.1, Express 5.2.1. - Master has an existing manifest/lockfile mismatch. Local installation used `--no-lockfile`. Express was restored to its locked version in an untracked test dependency directory. The tracked lockfile is unchanged. - The agent-skills suite stays below 400 MB of reported heap after the fix. - `git diff --check` passes. - Repository typecheck reaches the Runner Rust check and stops because this environment has no `cargo`. - The repository build and normal server typecheck also stop at the missing Rust toolchain. A direct server `tsc --noEmit` attempt ends with SIGKILL. - [CI run 38686964537](https://github.com/paperclipai/paperclip/actions/runs/38686964537) passes on `c744774bbada4e35ce3acf5991ff19cf50405f2f`, including all general and serialized server shards, typecheck, build, Runner checks, and e2e. - GitHub reports 53 successful check runs, two skipped Storybook checks, and a successful Snyk status. No checks are pending or failing. - [Greptile review](https://github.com/paperclipai/paperclip/pull/16396#issuecomment-6185194537) gives 5/5 on the same commit, with no actionable issues and zero review threads. - The full repository test suite was verified in CI; the local container has a 4 GiB memory limit. ## Risks Module state now lasts for one suite. Each case still constructs its own router and resets its mocks and fixtures. The affected suites pass together locally, and all applicable CI gates pass on the published commit. The change contains four test files. It changes no production code, workflow, manifest, or lockfile. Merge this separate fix before asking Dependabot to rebase #16017. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5bc4e878e7 |
fix: reuse verified seeds after warm workspace fallback (#16398)
Preserve the verified company/workspace seed-cache scope when a warm sandbox requires host-current staging. Keep remote identity checks, archive digests and rebuilding for missing or corrupt cache generations. 85 focused cases and full repository typecheck/build pass. Exact-head full CI/E2E, Superagent, legacy Snyk and Apex 5/5 pass with no unresolved findings. No staging response-time claim is made. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
79c34058cd |
fix(native): retain unexported workspaces after execution failure (#16397)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents can edit files in a remote sandbox before they submit a result. > - A process identity or startup failure can end the run before file export. > - Cleanup can then delete the sandbox and its unpublished files. > - This pull request retains the exact source and records a file repair action before cleanup. > - Operators can recover the files without replaying the failed turn or changing the task owner. ## Linked Issues or Issue Description **What happened?** A native remote process had completed local edits when process identity recovery failed. The run still had a prepared workspace reference and no accepted result. Cleanup deleted the source sandbox. No file recovery card appeared. A restarted run can hit the same path during setup, before the native executor starts. **Expected behavior** Retain the exact unexported source. Show the existing file recovery card. Require source stop and file recovery evidence. Keep task ownership, status, and external action reconciliation separate from file repair. **Steps to reproduce** 1. Run a native agent in a remote ephemeral workspace and make unpublished edits. 2. Fail process identity recovery before result acceptance or file export. 3. Allow failure handling and lease cleanup to run. 4. Before this change, cleanup can destroy the source without a file repair action. Refs: #16341 for related process identity diagnostics. Refs: #16349 for startup ownership fencing. This change addresses file retention after failure. ## What Changed - Record source retention and a file recovery action in the terminal native failure transaction. - Cover restarted preparation failures before the terminal setup writer and cleanup run. - Require the exact company, run, lease, provider allocation, and workspace binding. Require the exact live recovery claim to settle an owned preparation failure. Refuse stale or competing claims. - Reuse the existing stop-only cleanup and file-only reconciliation path. Leave accepted results and scheduled same-run recovery on their existing paths. - Document the retention rule and add database and runtime boundary regressions. ## Verification - All 1,242 tests passed across the full heartbeat recovery suite, native executor, native source retention, runtime teardown, run completion, and legacy file recovery. - The integrated missing-descriptor regression fails before the inner-handler fix and passes after it. The runtime retention and owned restart regressions also reproduce their prior failures. - The tests cover exact source identity, current and stale claims, atomic rollback, cancellation, idempotence, and file-only repair on a completed task. - `pnpm -r typecheck` passed on the final code. - `pnpm build` passed on the final code. - Full repository CI passed on `3945eb2ba5c228c5d4992da02b765dcf2e634cb8`: 54 successful checks and two skipped Storybook checks. - Apex scored 5/5 on that exact commit. Both earlier findings are fixed and all review threads are resolved. - The unsharded local `pnpm test:run` is still in progress. It is not counted as a local full-suite pass; complete suite coverage is verified in CI. ## Risks - Failed runs can retain a sandbox that needs operator cleanup after file recovery. This uses the existing retention policy and recovery controls. - This does not recover files from allocations that were already deleted. - No schema migration, UI change, or telemetry change is required. ## Model Used - OpenAI GPT-6 through Codex. Tool use, code execution, and reasoning assisted this change. The environment identifies the model as GPT-6; it does not expose a more specific serving snapshot or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
074c67fea8 |
fix(grok): avoid routine wakes for accepted work (#16394)
## Thinking Path
> - Paperclip lets people manage AI agents and their work.
> - Grok Bot receives assignments through routine webhooks and uses
agent MCP tools.
> - Each accepted assignment belongs to the current external
interaction.
> - A pending tool result or comment could start another routine while
that work was still active.
> - Quiet MCP traffic cannot prove that Grok has stopped working.
> - This pull request keeps receipts and feedback durable for the
current interaction to read.
> - New task routines start only for offers that the Bot has not
accepted.
## Linked Issues or Issue Description
Refs #16219 and #16305. The duplicate search also found #12991, which
covers instruction files for local Grok runs.
**What happened?**
The dispatcher could start another routine for an unread result after a
period without a routine reply. It could also wake accepted work for a
task comment. A queued delivery could still start after the Bot had
claimed the offer.
**Expected behavior**
Accepted work stays in its current interaction. The Bot polls the
original operation receipt and reads queued feedback at a safe boundary.
Acknowledgements of a sent wake remain valid after acceptance.
**Steps to reproduce**
1. Accept a Grok assignment and leave it active.
2. Add an unread comment, or complete a tool operation whose call
returned pending.
3. Run the dispatcher after its quiet period.
4. Observe another webhook call. Also queue an offer wake, accept the
offer before sending, and observe that delivery still starts.
**Paperclip version or commit**
Observed during real-client qualification on Core
|
||
|
DottaandPaperclip
|
854ab294cc |
fix: restore repository admission and Claude startup on Linux ARM64 (#16372)
Batch retained workspace admission reads while preserving individual privacy and trust decisions. Preserve bounded saved-policy projections when integrating current master, and qualify the pinned Claude executable on Linux ARM64.
Verified exact head
|
||
|
DottaandPaperclip
|
a15798e752 |
test(server): reuse seeded delivery workspace fixture (#16114)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
982ff62949 |
test(shared): consolidate ZIP conversion observations (#16118)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a05052f77d |
test: retire duplicate heartbeat disposition recovery executions (#16102)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
997ea5fd17 |
test(server): retire duplicate process-loss recovery execution (#16088)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aa6910939b |
test(runtime): share the projectless local lease acquisition (#16086)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d6e4b36c8c |
test(server): move Git redaction proof to its owner (#16081)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
76164b53cb |
fix(heartbeat): retry a legacy run that lost its lease before dispatch (#16358)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat runner executes legacy agent runs. A controller lease watchdog keeps durable ownership of each legacy run, and it aborts the run when it loses the lease. > - #16084 records a lease loss as a `controller_lease` cancellation when the adapter returns or throws. It does not cover a lease loss during setup, before the adapter starts. > - In setup, the abort reason goes to the setup catch. The catch records an opaque `setup_failed` and sends the raw `LegacyControllerLeaseLostError` to Sentry with no cause label. > - Issue continuation recovery treats `setup_failed` as non-retryable. The task moves to `blocked`, but no provider work started. > - This pull request records a lease loss before dispatch as `controller_lease_lost`, with the same `controller_lease` evidence. Recovery retries it as transient infrastructure, and so does the accepted-interaction (plan approval) continuation retry. > - The benefit is that a host lease loss during setup no longer strands the task, and the next Sentry event shows which renewal failure caused the loss. ## Linked Issues or Issue Description Refs #16084 Refs #16355 Refs #16359 **What happened?** A staging server reported `LegacyControllerLeaseLostError: Legacy controller lease lost` from the heartbeat. The run failed during setup with `setup_failed`. Continuation recovery then said "non-retryable failure on this issue's continuation run (`setup_failed`)" and moved the task to `blocked`. A scanner had to move the task back to `in_progress` by hand. **Expected behavior** A lease loss before adapter dispatch is a host infrastructure stop. The run records the `controller_lease` cause, like the adapter-phase path. The run keeps `providerWorkStarted: false`. Recovery retries the continuation with its bounded transient budget, and it does not block the task. **Steps to reproduce** 1. Start a legacy run that has a controller lease. 2. Before the adapter dispatch, make the lease renewal fail: the update matches no row, the query errors, or the lease deadline passes. 3. `controllerLease.assertOwned()` throws the abort reason in workspace preparation or before dispatch. 4. Before this change: the run is `failed`/`setup_failed`, `resultJson` has no `cancellation`, and continuation recovery blocks the task. After this change: the run is `failed`/`controller_lease_lost` with `cancellation.source: "controller_lease"` and a closed cause, and continuation recovery classifies it as `transient_infra`. **Additional context** #16355 handles a different path: a restart orphan that the next boot reaps as `process_lost`. This change handles a live controller that loses its own lease during setup. This change does not find the root cause of the lease loss. The closed cause label in the next Sentry event gives that fact. ## What Changed - `server/src/services/heartbeat/run-completion.ts`: `failRunSetup` receives the execution abort signal. When the abort reason is a `LegacyControllerLeaseLostError`, it writes error code `controller_lease_lost`, adds the `controller_lease` cancellation evidence, and keeps the existing bootstrap recovery facts. The agent goes back to idle, not `error`, because the stop is a host fault. This applies only before provider dispatch: `failRunSetup` also receives `providerDispatchStarted`, and a setup-path error after a legacy adapter or native dispatch began keeps `setup_failed`, so started work is never retried. - `server/src/services/legacy-controller-lease.ts`: add the `CONTROLLER_LEASE_LOST_FAILURE_CODE` constant. - Merged with #16359 (lease-loss timings): the setup failure keeps its `resultJson.controllerLeaseLoss` timings and adds this PR's `controller_lease` cancellation from the abort signal. #16359's setup test now aborts the signal first, as `assertOwned()` does, and expects `controller_lease_lost`. - `server/src/services/heartbeat.ts`: pass the execution abort signal and `providerDispatchStarted` (`legacyAdapterEntered || nativeDispatchStarted`) to `failRunSetup`. - `server/src/services/recovery/service.ts`: add `controller_lease_lost` to the transient-infrastructure continuation codes (3 bounded attempts with backoff). - `server/src/services/heartbeat/retries.ts`: add `controller_lease_lost` to the accepted-interaction continuation retry allowlist, next to `process_lost`. Without it, a plan-approval continuation that lost its lease before dispatch was escalated and blocked before generic recovery could see it. - `server/src/services/heartbeat/run-lifecycle.ts`: add `controller_lease_lost` to the pre-adapter setup codes, so the comment policy does not queue a missing-comment retry. - `doc/run-log-events.md`: document the new error code. - Tests: setup lease loss when the abort reason is thrown, and when a setup step throws a different error after the abort. A unit test shows a lease loss after dispatch keeps `setup_failed`. A classifier test shows `controller_lease_lost` is transient and `setup_failed` stays non-retryable. Two assembled heartbeat tests (`heartbeat-process-recovery.test.ts`) lose the real lease during workspace preparation: one for a plain issue continuation (one `transient_failure` retry) and one for an accepted plan-approval interaction continuation (one `interaction_continuation_infra_retry` retry). Both check the saved run, no adapter dispatch, and the issue still `in_progress`. ## Verification - `cd server && npx vitest run src/services/heartbeat/run-completion.test.ts`: 25 passed (embedded Postgres), including the new post-dispatch test. - `cd server && npx vitest run src/services/run-cancellation.test.ts src/services/recovery/provider-failure-classification.test.ts`: passed on the first head (29 with run-completion; 25 classifier). - `cd server && npx vitest run src/services/legacy-controller-lease.test.ts`: 13 passed (needs `pnpm --filter @paperclipai/plugin-sdk... build` first). - After merging master (head `2148b27`): `run-completion`, `legacy-controller-lease`, `run-cancellation`, `provider-failure-classification` and `run-failure-report` suites, 310 passed; `heartbeat-process-recovery.test.ts -t "lease|dispatch"`, 31 passed. - Head `c52825c`: `retries`, `run-completion`, `heartbeat-retry-scheduling`, `legacy-controller-lease` and `provider-failure-classification` suites, 134 passed; `heartbeat-process-recovery.test.ts -t "lease|dispatch|interaction"`, 41 passed. - Negative controls: with the lease-loss branch in `run-completion.ts` disabled, the plain-continuation assembled test fails (`setup_failed` instead of `controller_lease_lost`). With the `retries.ts` allowlist change reverted, the accepted-interaction assembled test fails because no retry is scheduled. - Not run locally: full `tsc` and `pnpm build`. The sandbox has a 4 GB memory limit and the OOM killer stops `tsc`. CI is the typecheck and build gate. ## Risks - Low risk. The change applies only when the execution abort reason is a `LegacyControllerLeaseLostError` in the setup catch before dispatch. Other setup failures keep their current codes. - The retry uses the existing continuation recovery budgets and admission checks (agent state, budget, pause). The interaction-continuation retry keeps its existing 2-attempt budget. The retry is a new run. The revoked controller cannot dispatch, because its renewal compare-and-set fails. - A run that the reaper settles first keeps the reaper outcome. `setRunStatusIfRunning` makes the first terminal write win. - Sentry still gets this unexpected stop, with the closed `controllerLeaseLossCause` label. If the lease loss is frequent, the event volume does not change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (targeted suites pass; typecheck/build not possible in this sandbox, see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
3ea39f75c1 |
test: consolidate duplicate owner proofs (#16075)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
5f160421e3 |
test(server): run status corpus fixtures once (#16078)
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
8b27af77db |
fix(heartbeat): wait for a live holder of a reused shared execution workspace (#16371)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat pre-dispatch gate decides if a run can use a shared workspace in its execution environment. > - A child task can inherit the shared execution workspace of its parent. The parent run is often still live when the child run starts. > - On a sandbox environment, the gate failed the child run with `configuration_incomplete`. It did this for every concurrency policy, also for the default `auto` policy. > - Recovery then blocked the task and told the operator to bind missing secrets. That was not the real cause. An operator had to select a different workspace by hand. > - Shared project workspaces already use the `auto`/`serialize` policy to wait through the `workspace_busy` retry path. > - This pull request applies the same policy to a reused shared execution workspace. > - The benefit: the child task waits for the parent run and then continues automatically. It does not fail and become blocked. ## Linked Issues or Issue Description **What happened?** A parent agent created a child task. The child inherited the parent's shared execution workspace. The child agent used a sandbox environment. The child run started about 1.5 seconds after the parent created the task, while the parent run was still live. The run failed before the adapter started with this error: "This environment cannot place concurrent agents in one physical shared folder. Use local/SSH sharing or isolated task workspaces." Recovery set the task to `blocked` with the message "required secret/env bindings are missing". **Expected behavior** With the default `auto` policy, the child run waits for the parent run (`workspace_busy`) and then continues automatically. This is the same behavior as for a shared project workspace on a sandbox. **Paperclip version or commit** Staging release `8ffc5748`. The same code is on `master`. **Deployment mode** Hosted cloud (staging), agent on a sandbox execution environment. **Steps to reproduce** 1. Use an agent with a sandbox environment. 2. Bind two issues to the same `shared_workspace` execution workspace. Set `executionWorkspacePreference: reuse_existing` on the second issue. 3. Keep a live run on the first issue. 4. Wake the agent on the second issue. Before this change, the run fails with `configuration_incomplete`. After this change, the run is deferred with `workspace_busy`. ## What Changed - `environment-selection.ts`: When a reused shared execution workspace has a live holder on a non-local/non-SSH environment, the gate now reads `sharedWorkspaceConcurrency`. With `auto` or `serialize`, it throws `WorkspaceBusyDeferral`. Only `allow` still fails with `shared_realization_unsupported`. - `retries.ts`: Added `findExecutionWorkspaceHolder` next to `findSharedWorkspaceHolder`. It keeps the old match rule. A running holder that has had no output for more than one hour (the existing staleness cutoff) is still returned, with `stale: true`. A live holder is preferred over a stale one. - `environment-selection.ts`: A stale holder still owns the folder, because silence does not prove that the run stopped. The child run therefore fails with `configuration_incomplete` (`shared_realization_unsupported`, `holderStale: true`). It does not share the folder with the holder, and it does not wait without limit. This is the same result as before this pull request, with a more specific message. - `retries.ts`: `WorkspaceBusyDeferral` can now carry an `executionWorkspaceId`. The run result, the run event, and the cancellation reason name the execution workspace when it applies. - `stranded-notice.ts`: Added a specific recovery notice for `shared_realization_unsupported`, and a second notice for a stale holder. They no longer tell the operator to bind missing secrets. - `doc/experimental/issue-worktree-support.md`: Documented that `allow` is only concurrent on local/SSH, and that a reused shared execution workspace follows the same rules. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-workspace-busy.test.ts -t "reused shared execution workspace"` covers 4 tests: - `auto` on a sandbox with a live holder: the run is cancelled with `workspace_busy`, a retry is scheduled, no Sentry run failure is reported, and the issue does not become blocked. - After the holder finishes, the scheduled retry is promoted and runs. The adapter gets the inherited shared folder, and the issue keeps its execution workspace. - `allow` on a sandbox: the run still fails with the shared-folder message and no retry is scheduled. - A holder that is still running but silent past the staleness cutoff: the run fails with `holderStale: true`, the adapter does not start, no retry is scheduled, and the holder is not touched. - `npx vitest run src/services/recovery/stranded-notice.test.ts src/services/heartbeat/environment-selection.test.ts src/services/heartbeat/retries.test.ts` covers the notice and the selection unit tests. - First commit (`d0b7a092`): the 3 original integration tests and the unit suites passed locally, and all CI checks passed. - Follow-up commit (`3c7e515`, which addresses the Greptile review): these tests were **not** run locally. Dependency install was killed twice by the sandbox's 4 GB memory limit. CI is the first run of the new and changed tests. - Known local failure: `does not hide ambiguous, malformed, provider, or transport cancellations behind workspace_busy` fails in this sandbox also on unchanged `master` (checked with `git stash`). This change does not cause it. - Not run locally: the full server `tsc --noEmit`, for the same memory reason. CI runs the typecheck. ## Risks - Behavior change: before, a run with a live holder of a reused shared execution workspace on a sandbox failed at once. Now it waits with `auto`/`serialize`. A long parent run can delay the child. If the holder has no output for more than one hour, the child fails as before instead of waiting without limit. - An explicit `allow` policy keeps the old failure. - Local and SSH environments do not change. - No schema or migration change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in the Paperclip agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (first commit only; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
66724b8ce0 |
perf(auth): project only required run context fields (#16380)
Project only the required JSON fields for saved-run policy and direct-parent report authorization. Preserve current reads, JSON types, identity checks, permission decisions and legacy task-key precedence. Validation: 186 affected cases; 17 database transfer-budget cases after the shared startup-timeout correction; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1d5e2c89f0 |
test(server): retire duplicate guards and share issue seed fixtures (#16059)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Server tests protect task scheduling, session continuation, transport decisions, and issue data. > - Four cases repeat the same registered inputs and assertions as existing owner cases. > - Issue service tests also repeat 103 database seed statements. > - Each distinct input, assertion, lifecycle, and security contract must keep its proof. > - This pull request removes the four exact duplicates and shares only exact seed statements. > - The benefit is 584 fewer test lines and four fewer registered executions. ## Linked Issues or Issue Description Refs #16047. This draft is stacked on `codex/test-audit-round-six-delivery` at `e868e6645ce0d598d756d3458f0b3d71afd6db4a`. **What existing behavior does this improve?** Maintenance of server owner tests. Production behavior does not change. **Current behavior** Four registered cases repeat identical executable bodies. The issue service suite repeats 80 company rows, ten project rows, and thirteen agent rows. **Proposed behavior** Keep each contract in its existing owner suite. Use three direct async seed helpers for the exact repeated database rows. Preserve each caller's database instance, IDs, awaited insert position, field order, and fresh object allocations. **Reason and benefit** Remove 584 net test lines and four registered executions. No measured wall-time improvement is claimed. No production, dependency, workflow, telemetry, or skip change is included. ## What Changed - Share exact company, Workspace project, and CodexCoder rows in `issues-service.test.ts`. All 137 registered cases remain. - Retire `heartbeat-scheduling-suppression.test.ts` → `suppresses heartbeat scheduling while a task drain is active`. Keeper: `heartbeat-task-drain.test.ts` → `start_task_drain_suppresses_admission`. - Retire `heartbeat-timer-wake-session-reset-pf4.test.ts` → `does not reset for issue_commented (preserve continuation context)`. Keeper: `heartbeat-workspace-session.test.ts` → `does not reset for comment wakes`. - Retire `setup-token-session.test.ts` → `denies a direct non-loopback HTTP receive-token request, so it delivers no token (SR-6)`. Keeper in the same describe: `denies a direct non-loopback HTTP request (SR-6)`. - Retire `setup-token-session.test.ts` → `denies a spoofed forwarded HTTPS receive-token request under a broad proxy setting (SR-7)`. Keeper in the same describe: `denies a spoofed X-Forwarded-Proto when no trusted proxy is configured (SR-6)`. The retired receive-token cases only call `evaluateConfidentialTransport`. They neither request a route nor configure `TRUST_PROXY`. The keepers protect the exact guard inputs and output. The route now uses the guard to select a transport advisory; this PR makes no route-denial or token-delivery proof claim. ## Verification - Complete matched six-file baseline: **414/414 pass**, no failures or skips. Candidate: **410/410 pass**, no failures or skips. - Run: `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/issues-service.test.ts server/src/__tests__/heartbeat-scheduling-suppression.test.ts server/src/__tests__/heartbeat-task-drain.test.ts server/src/__tests__/heartbeat-timer-wake-session-reset-pf4.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts server/src/services/setup-token-session.test.ts`. - Expand the actual three helper bodies at all 103 call sites. The complete issue-service AST is identical to the original after expansion. Independent review checked the exact inputs, lexical bindings, awaited order, and fresh nested configs. - Deliberate regressions disabled task-drain suppression, forced comment wakes to reset, and allowed the final insecure transport branch. All four retained keepers failed at their intended equality assertions. Each production file was restored byte for byte against HEAD. - An optional project-inheritance mutation was ineffective because another workspace path still derives project identity. It passed and is excluded from accepted preservation evidence. - `git diff --check` passes. Independent cross-review approved the exact source and proof before this draft. - Broad typecheck, full test, and build gates passed in PR CI on `a9713432b1e498e023b4828ffa283a351abe856f`. All 52 check-runs concluded success. Greptile completed on that exact head with confidence 5/5 and no findings. ## Risks - Low risk: only test code changes. Four exact duplicate bodies are removed. - The fixture helpers preserve all original IDs and DB operations. They do not cache data or generate assertions. - Same-named explicit-project and derived-project inheritance cases remain because their inputs differ. Separate login PTY and duplex lifecycle cases also remain. - No wall-time speed gain is claimed from fixture line reduction. ## Model Used - OpenAI GPT-6.1-Sol, exact model ID `gpt-6.1-sol`, through Codex. Tool use, code execution, and reasoning assisted the audit. The runtime does not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d73d7a8651 |
perf(skills): batch repository dependency reads (#16377)
Batch dependency Markdown reads through the bounded Git snapshot reader and queue newly discovered dependencies once. Preserve package contents, limits, authorization, audits, and reference order. Validation: 63 affected cases; full typecheck/build; exact-head full CI, E2E, Superagent and Apex 5/5. Local Git comparison: 502 individual reads reduced to three batches with identical entries. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
275189d8d6 |
fix(native): report why an unclassified Codex turn failed (#16354)
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs use the Paperclip runner to drive a provider session, for example the Codex app server > - When a Codex turn fails, two classifiers look for known failures: a model rejection and a server capacity failure > - When no classifier matches, the run fails with the generic message `Native session failed` and the error code `adapter_failed` > - The run failure report to Sentry then has no data about the cause, because the server drops the Codex `codexErrorInfo` value > - This pull request records a closed-vocabulary diagnostic for that failed turn and sends it to the Sentry `provider_failure` context > - The benefit is that operators can see which Codex failure kind causes these generic failures, and then write a targeted fix ## Linked Issues or Issue Description **What happened?** Hosted staging reports many run failures with the title `Error: Native session failed`. The events have `agent_adapter=paperclip_runner`, `error_code=adapter_failed`, `driverKind=codex_app_server`, and `exitCode=1`. The events do not show why the Codex turn failed. **Expected behavior** A run failure report for a failed Codex turn shows the failure kind, so that an operator can find the cause. The report must not contain provider response text, prompts, or model names. **Steps to reproduce** 1. Start a native Codex run. 2. Make the Codex app server end the turn with `status: "failed"` and an error that is not a model rejection or `serverOverloaded`. For example, use `codexErrorInfo: "contextWindowExceeded"`. 3. Look at the Sentry event for the run failure. Before this change, the `provider_failure` context is empty. **Agent adapter(s) involved** Codex (native runner, `codex_app_server` driver). ## What Changed - `native-provider-failure.ts`: The provider failure observation now records a diagnostic for each failed Codex turn. The diagnostic has only closed values: `provider: "codex"`, `category: "turn_failed"`, a known `CodexErrorInfo` kind (or `unrecognized` / `absent`), an HTTP status from 100 to 599 for the HTTP variants, and a runner failure code from an explicit allowlist of the codes the runner itself writes on a failed Codex turn (any other string becomes `unrecognized`, because a provider error object can carry its own `code`). It never copies the provider message. - New `unclassifiedTurnFailure()` gives this diagnostic only for a failed terminal turn that `forTerminal()` does not classify. `forTerminal()`, error codes, error messages, retries, and finalizer decisions do not change. - New `readNativeTurnFailureDiagnostic()` rebuilds a persisted value from the closed vocabulary and rejects all other values. - `native-session-executor.ts`: The adapter result keeps the generic message. It adds `resultJson.nativeTurnFailure` when the turn failure is unclassified. - `run-failure-diagnostics.ts`: The run failure report copies the validated diagnostic into the `provider` context. Sentry sends this as `provider_failure`. - Tests: unit tests for the vocabulary, HTTP status bounds, unknown kinds, runner codes, and privacy; a diagnostics test; and an update to the end-to-end executor test for the `unknown-error` scenario. ## Verification - `pnpm exec vitest run src/services/native-runtime/native-provider-failure src/services/__tests__/run-failure-diagnostics.test.ts src/services/__tests__/run-failure-report*.test.ts src/__tests__/sentry*.test.ts` (in `server/`): 6 files, 432 passed, 5 skipped. - `pnpm exec vitest run src/services/native-runtime/native-session-executor.test.ts` (in `server/`): 698 passed. - I could not run the full `tsc --noEmit` in my sandbox: the memory limit stopped the process, and the runner vendor build needs `cargo`. CI runs the full typecheck. - To check by hand: make a Codex turn fail with an unclassified error. Then make sure that the Sentry event `provider_failure` context shows `category: turn_failed` and the `codexErrorInfo` kind, and no provider text. ## Risks - Low risk. The change adds diagnostic data only. Run status, error code, error message, retry, and finalizer logic do not change. - Privacy: the diagnostic uses an allowlist of Codex enum names, a bounded integer, and an allowlist of runner-issued failure codes. Tests make sure that provider text does not get through. - If Codex adds a new `CodexErrorInfo` kind, the diagnostic shows `unrecognized` until the allowlist gets the new kind. - This change does not fix the root cause. It gives the data that is necessary to find the cause. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Related: #15347 (capacity classifier), #15304 (model rejection diagnostics). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3a440d28c4 |
fix(server): read existing grants before seeding role defaults (#16365)
`insertMissingPrincipalGrants` now reads the existing permission keys first and inserts only missing grants. In the steady state no INSERT runs, so cloud-tenant auth resync no longer waits on row locks held by concurrent grant edits until the statement timeout (Sentry PAPERCLIP-BACKEND-3S). Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ccc77c5b5e |
fix(server): record legacy controller lease-loss timings (#16359)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat runner executes legacy agent runs, and a controller lease watchdog keeps durable ownership of each run > - The watchdog renews the lease every 10 seconds and stops the run when a renewal does not finish within the 60-second lease > - A staging run failed in setup with `LegacyControllerLeaseLostError` and the `renewal_deadline` cause, after about eight minutes of preparation and with no server restart > - The setup failure path did not record the cause, and no evidence showed why the renewal did not finish: a stalled database query or a blocked event loop > - This pull request records bounded watchdog timings with each lease loss, on the setup failure path and on the cancellation path > - The benefit is that the next event shows which of the two causes stops renewal, which is the missing fact to fix the root cause ## Linked Issues or Issue Description Refs #16084 (added the closed lease-loss cause for cancelled runs) Refs #16355 (retries a run that a restart stopped before dispatch; a different path) **What happened?** A legacy `paperclip_runner` run on staging failed with `setup_failed` and the message "Legacy controller lease lost". The stack shows the watchdog deadline timer (`renewal_deadline`). The run was in `executionStage: "preparing"` for about 467 seconds. The server started about 11 minutes before the run, so this was not a restart. The run record and the Sentry event do not show the lease-loss cause, because the setup failure path does not read it. Nothing shows if a renewal query was in flight at the deadline, or if the deadline callback ran late. **Expected behavior** When the watchdog loses the lease, the run records the closed cause and the watchdog timings. This applies when the loss fails setup and when it stops the adapter. The failure report contains only the closed cause and bounded numbers. **Steps to reproduce** 1. Start a legacy run with a controller lease. 2. During setup, make a lease renewal query hang for more than 60 seconds, or block the event loop past the deadline. 3. Let the watchdog abort the run. 4. Read `resultJson` on the failed run. Before this change, it has no lease-loss evidence. ## What Changed - `legacy-controller-lease.ts`: The watchdog tracks the last successful renewal, the renewals in flight, the scheduled deadline, and the renewal count. `LegacyControllerLeaseLostError` now carries `leaseLossTiming`: `sinceRenewalMs`, `renewalPendingMs` (or `null`), `deadlineLateMs` (only for `renewal_deadline`), and `renewalCount`. Ownership behavior does not change. - `run-cancellation.ts`: Add the `LegacyControllerLeaseLossTiming` type and `readControllerLeaseLoss()`. The reader accepts only a closed cause and non-negative finite numbers, and clamps each value to 24 hours. - `heartbeat/run-completion.ts`: Persist `resultJson.controllerLeaseLoss` with the cause and timings. This applies to a lease loss that stops the adapter and to a lease loss that fails setup (the thrown abort reason). - `run-failure-diagnostics.ts`: Report `controllerLeaseLossCause` and the four bounded timing fields from `controllerLeaseLoss`. A setup failure now gets the cause label too. - `doc/run-log-events.md`: Document the new record and how to read the timings. - Tests: watchdog timings for a stalled query, a blocked event loop, and a failing query; completion records the evidence for an adapter stop and for a setup failure; the failure report clamps values, drops unknown keys, and drops a record without a closed cause. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/services/legacy-controller-lease.test.ts src/services/heartbeat/run-completion.test.ts src/services/__tests__/run-failure-report.test.ts src/services/run-cancellation.test.ts` - I could not run these tests locally. The sandbox has a 4 GB memory limit, and `pnpm install` was stopped by the out-of-memory killer during lockfile resolution. CI is the test run for this change. - After deployment, a new lease loss shows `controllerLeaseRenewalPendingMs` and `controllerLeaseDeadlineLateMs` in the Sentry execution context. A pending time near 50 seconds or more points at the database. A large deadline lateness points at the event loop. ## Risks - Low risk. This change adds diagnostics. It does not change when the watchdog stops a run, and it does not prevent the lease loss. - The new `resultJson.controllerLeaseLoss` field is server-written. Readers ignore unknown values. Old rows do not have it. - The failure report gets five closed or numeric fields. It gets no reason text, IDs, or user content. - The event-loop test blocks the loop for about 80 ms to make the deadline late. It uses real timers, so a very slow CI host makes the lateness larger, not smaller. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, Sentry and GitHub connectors). The exact context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (not possible in this sandbox; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
3dffd2197d |
test: consolidate owner fixtures and retain real regression proof (#16047)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tests protect sandbox execution, run recovery, credentials and the board UI. > - Several owner suites repeat the same setup and prove the same result more than once. > - These copies add maintenance work and can hide weak assertions. > - Each cut needs a retained test that proves every distinct contract. > - This pull request consolidates repeated setup and removes twelve redundant executions. > - It also replaces two weak assertions with checks of real outputs. > - The benefit is 2,831 fewer test lines with clearer regression proof. ## Linked Issues or Issue Description **What existing behavior does this improve?** The maintenance and regression strength of the existing Vitest owner suites. **Subsystem affected** Sandbox adapter utilities, server heartbeat recovery and environment capabilities, and the board UI. **Current behavior** Tests copy database rows, query builders, render calls and runtime inputs. Some cases repeat an existing contract. One token test omits a real bridge credential. A PostHog host comparison can compare absent controls. **Proposed behavior** Use small local helpers with fresh values and the same action order. Keep each distinct contract in an owner test. Check real bridge credentials and fixed PostHog method states. Carry the exact persisted first-run success assertion into the stronger cancellation keeper. **Reason and benefit** The ten test files shrink by 2,831 net lines. Twelve redundant executions retire. All other case tables, inputs, security boundaries, timeouts and cleanup rules remain. **Breaking changes** None. This round changes tests and audit evidence only. **Additional context** Refs: #15978, the merged first cleanup. Refs: #16022, the merged prior cleanup. Its squash merge is `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Related open work: #14634 repairs feedback fixture completion; #14477 supplies explicit sandbox fixture tools; #14198 changes cleanup retry behavior in another heartbeat file. This audit preserves those separate action and cleanup contracts. It does not apply their changes. This PR was first stacked on `codex/test-audit-followup`. After #16022 was squash merged, only the two Round 6 commits were replayed onto merged master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. The original stack head is preserved in a local snapshot branch. The current diff contains only ten test files and two evidence files. No merge is requested. ## What Changed - Consolidate repeated setup in three sandbox adapter suites. Remove seven redundant executions and retain the real protocol, cleanup and scratch-file checks. Save 875 lines. - Consolidate heartbeat database fixtures and read queries. Retain exact wake cancellation, total run count and the actual first-run success receipt. Remove one duplicate execution. Save 1,372 lines. - Remove two repeated rejected-capability cases. Carry the absent snapshot assertion into the retained runner case. Save 22 lines. - Consolidate UI fixtures and render calls. Remove two repeated connection cases. Replace the weak PostHog comparison with fixed method and required-key checks in both real hosts. Save 562 lines. - Record audit scope, decision and expansion counts, all twelve keeper mappings and deliberate mutation results in the [audit report](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6.md) and [controls evidence](https://github.com/paperclipai/paperclip/blob/codex/test-audit-round-six-delivery/doc/plans/2026-10-12-test-audit-round6-controls.json). ## Verification Source baseline: `a090035cafb29c06f6fca2dbe43561ad866b7c1b`. All ten original test programs are byte-identical at the stacked base `58c5c4aa241c85f4665a3159c192888bbb2a5cfa`. - Baseline: 1,841/1,841 cases passed in the main thirty-file owner/sibling run. Another 219/219 cases passed in nine extra files. - Preservation: all four proposal packages passed independent review. Whole-program helper expansion retains ordered inputs, callbacks, awaits, queries, fresh allocations and hooks outside the declared assertion migrations and repairs. - Candidate: all 2,048 cases passed across 39 complete owner/sibling files. The main run passed 1,831 cases. The extra run passed 217 cases. Both runs had zero failures and zero skips. - PostHog control: the repaired positive case passed. The original weak case passed with authentication choices removed. The repaired case failed at its fixed OAuth aria-checked assertion under the same mutation. - Eight accepted mutations failed at their intended assertions. They cover bridge-token exclusion, fallback cleanup, READY bytes, real scratch-file preservation, exact wake cancellation, the persisted first-run success receipt, rejected capability snapshots and reachable PostHog method controls. Two original weak cases passed with the same defects. All production source bytes were restored. - Rejected control: changing run status before legacy release timed out before wake cancellation. This result is not counted as assertion proof. Its replacement changed only the persisted receipt after real drain in the existing transaction. It failed at the direct fixed succeeded-row assertion. - No new code comments or 1,000-line threshold crossings were introduced. No production, schema, lockfile or workflow changes are included. On the exact final test sources, later committed as `d8de9422f0703d4508a9dc880b67a07f13e0f110`, `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build` passed. The required full `pnpm test:run` completed with **exit 1** after **52.14 minutes** on clean, immutable `d8de9422f0703d4508a9dc880b67a07f13e0f110`. HEAD and tracked sources remained unchanged. The first general-server group ran 889 files: **882 passed, one failed and six skipped**. Its 18,032 cases were **17,925 passed, two failed and 105 skipped**. The only failing file was `server/src/__tests__/remove-test-home.test.ts`. On macOS, the read-only runtime bundle and symlink cleanup cases returned `ENOTEMPTY`. The permission-error case expects `EACCES` or `EPERM`. Four reported failure entries cover these two cases, including their cleanup errors. The test and helper are byte-identical to previous master `4b766b872993045a2e787404f0475503683e5ded` and audit master `e364b92c82`. The default wrapper stopped after that group failed. **Later local groups did not run.** The earlier public-MCP, HTTP log redaction and workspace snapshot failures did not recur. The 39 focused owner/sibling files and all 48 remote CI jobs completed independently. This result is an unresolved macOS cleanup limitation, not full local success. No test, skip, retry, timeout or cleanup rule was changed to hide it. The three adapter files received a whitespace tidy after focused execution. Full runtime AST and string/template/regex byte equality link those tested sources to the final sources. All 134 adapter helper calls were checked again. Replayed code commit `3d3defd2a60595a623045d9e2c6c586f4acb3452` has the exact final test bytes. Evidence commit `e868e6645ce0d598d756d3458f0b3d71afd6db4a` adds the report and controls JSON, with one historical ancestry clarification. All ten accepted test hashes and all seven production-owner source guards match. No package, lockfile, workspace, TypeScript, Rust or Vitest dependency manifest changed since the audit snapshot. The [original stack CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38312876910) passed all 48 jobs on head `c171d6c0d603d22bb997c1f9a9bdcd5bcc5d22aa`. Those results are historical stack proof. The [fresh master-context CI workflow](https://github.com/paperclipai/paperclip/actions/runs/38318223856) completed successfully with **all 48 jobs passed** on current head `e868e6645ce0d598d756d3458f0b3d71afd6db4a`. Its tested merge commit was `41292af60c6eb60d477637d23822bb17b08dc07e`, combining workflow base `88bf4c6c1d222f818faa56257efe79207c892a6c` with that head. The delivery commits remain based on pinned master `dabb8c5daf430a1987af611e2e7ca4c7b23ef7d4`. Both required master checks, `ci / verify` and `ci / e2e`, passed. Two separate optional Storybook checks were skipped by their scope gate. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/16047#issuecomment-6133323165) gives the current head **5/5**, with no findings, inline comments or unresolved review threads. The [current-head commitperclip review](https://github.com/paperclipai/paperclip/actions/runs/38318223666) also completed successfully. This PR remains open and draft; no merge is requested. The local macOS failure above remains unresolved. ## Risks - A helper could share a value or capture the wrong binding. Independent expansion and lexical binding checks guard the actual helper calls. - A retired case could contain a distinct assertion. Each cut has an exact keeper map and carried observations. - A mutation could fail before the intended assertion. Controls reject timeouts and earlier failures. The rejected receipt mutation remains in the evidence. - The delivery ancestry changed after the parent squash merge. Exact source hashes preserve the audited tests and producer receipts. Fresh CI and review passed in the current master context; the broad local run remains evidence from its immutable audit snapshot. - The complete local default failed in two unchanged macOS cleanup cases. Later local groups were not run. This limitation remains visible even though focused checks and remote CI passed. ## Model Used OpenAI Codex, based on GPT-6, assisted with this work. The exact served model ID and context window are not exposed in this session. It used repository tools, code execution, AST analysis and parallel audit agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
cafa25029c |
fix(native-runtime): recover unclaimed remote startup (#16349)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The native runtime preserves task execution across server restarts. > - Remote workspace preparation can finish before the executor claims its first provider attempt. > - The recovery classifier required a provider checkpoint even when the durable coordinator proved that execution had never started. > - A deployment during this interval therefore left safe startup work on a permanent recovery hold. > - This pull request admits that narrow startup state to the existing same-run bootstrap path. Existing execution and ownership checks remain in force. > - The original preparation yields when recovery owns the coordinator. It cannot fail or clean up the recovered run. > - Tasks can continue after this interruption without an operator retry or a duplicate provider turn. ## Linked Issues or Issue Description **What happened?** A server deployment interrupted a prepared remote workspace before the native executor's first claim. The coordinator had attempt zero, controller generation zero, and no owner, process, provider event, or checkpoint. Restart classification returned `ambiguous_remote_recovery_evidence` and blocked the task. **Expected behavior** The server should recognize this durable never-claimed state and continue the original run through normal execution admission. A prior claim, uncertain ownership, or contradictory execution evidence must retain the recovery hold. **Steps to reproduce** 1. Prepare a remote workspace for a native run. 2. Stop the controller before the executor commits its first claim. 3. Restart the controller and classify the running native task. 4. Observe that the old classifier blocks it because no provider checkpoint exists. Related: #16341 preserves remote recovery failure reasons. #16344 strengthens remote restart qualification. Neither changes admission for an untouched remote startup. ## What Changed - Require a server-owned fencing version written only with fresh native identities. Older unmarked preparations stay held during rolling upgrades; caller profile data cannot add that marker to an existing run. - Leave the current server's active preparation with its existing execution. - Yield through the controller-detachment path when a different execution owns the coordinator lease. Lease expiry does not grant takeover authority. - Preserve run-scoped gateway tokens during detachment so stale preparation cannot revoke the recovered worker's tools. - Admit a valid prepared remote workspace only when the locked coordinator proves attempt and controller generation zero, no owner, and no prior session or execution evidence. - Check launch receipts and provider source events before granting bootstrap recovery. - Keep the original run and session. Preserve atomic recovery claims, executor durable-state checks, and normal admission gates. - Add real PostgreSQL regression tests for competing controllers and seven contradictory evidence cases. - Update recovery documentation, the run-log reason, and both capability maps. ## Verification - The competing-controller regressions failed before the ownership fix and pass for both live and expired foreign leases. - On the final revision, 77 runtime-selection, restart recovery, real PostgreSQL, and heartbeat boundary tests pass with no skips, including all local runner process fixtures. The full native executor file passes all 698 tests. - Another 91 live-run route tests pass locally. An earlier CI timeout in that unrelated file passed on retry; the complete final-head CI passed without that failure. - `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check`: passed locally. The map check verifies structure only. - Local `pnpm test:run` did not finish: an early attempt had build-related collection failures, and later full-suite execution was stopped after complete CI passed. Targeted suites passed after build preparation. This is not a local full-suite pass. - The complete [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/38669996667) passed on `2baffb3652bfeb4705a30ca3706fbda6c3d72aa4`, including all test shards, build/typecheck, runner verification, and browser E2E. All 55 applicable review/security/CI checks succeeded; two Storybook checks were skipped. - [Greptile Apex](https://github.com/paperclipai/paperclip/pull/16349#issuecomment-6180848957): 5/5 on the same final head, with no unresolved findings. - No live remote provider restart is claimed by the database regression. ## Risks - A bare executor can no longer take a foreign coordinator lease after expiry. A verified recovery claim must transfer authority first. - Existing preparations without the server-owned fencing marker retain their recovery hold during rolling upgrades. - Incorrect admission could duplicate provider work. This path requires a positive durable never-claimed receipt, rejects contradictory launch evidence, and keeps the executor's existing durable-root verification. - Previously claimed or incomplete remote recovery evidence remains blocked. This change does not clear historical terminal holds. - No schema, API, dependency, or UI changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, code execution, and browser inspection. The exact serving snapshot ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
DottaandPaperclip
|
ff0f976de1 |
fix(native-runtime): retain remote recovery failure reasons (#16341)
Preserve closed marker-read and process-verification reasons through native recovery failures and operator reports. Keep existing transport errors and reattachment ownership checks. Add shell, process-probe, persistence and privacy regression coverage. All current-head CI checks pass and Greptile is 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
1b58ac6ec9 |
build(deps): bump smol-toml from 1.8.0 to 1.9.1 (#16020)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Codex runner must set trust for the exact startup project. > - smol-toml 1.9.1 returns tables with null prototypes. > - Object spread replaces those prototypes and causes strict validation to reject valid trust edits. > - This PR keeps the parsed tables and adds tests for the dependency update. > - Codex can start while unrelated settings and trust boundaries stay intact. ## Linked Issues or Issue Description Refs #16028. That merged PR contains the same table-preservation fix and tests for both prototype styles. This branch retains that work and adds more regression cases. Related dependency updates: #15293 and #15364. **What happened?** Eight startup trust tests fail after the update to smol-toml 1.9.1. The error is `codex_startup_trust_cannot_preserve_configuration`. **Expected behavior** Set trust for the exact startup project. Keep unrelated configuration bytes and sibling project trust unchanged. **Steps to reproduce** 1. Check out the original PR head, `90b133dfd963af4f19b4df11fdd6c11b4eeb2a2d`. 2. Install dependencies with `pnpm install --frozen-lockfile`. 3. In `packages/paperclip-runner`, run `pnpm exec vitest run src/drivers/codex/codex-startup-trust.test.ts`. **Paperclip version or commit** The verified head is `1a8e8ad8b2e1049ebb68595d85c92e3c0621559d`. **Deployment mode** Built from source on Linux. The affected code runs before Codex starts. ## What Changed - Keep Dependabot's update from smol-toml 1.8.0 to 1.9.1 in the runner and server. - Keep parsed project tables. Use `parse("")` to create missing tables with the parser's prototype. - Keep strict candidate validation and the upstream tests for both prototype styles. - Add tests for nested values, dates, times, special numbers, prototype-named keys, sibling trust, and repeated calls. - Preserve the original Dependabot commit and lockfile. The two follow-up commits change only the trust source and tests. ## Verification - All 28 startup trust cases pass at the verified head. Run the command above. - The focused strict TypeScript check passes. Use `tsc --ignoreConfig --noEmit --module nodenext --moduleResolution nodenext --target es2023 --types node --esModuleInterop --strict --skipLibCheck src/drivers/codex/codex-startup-trust.ts src/drivers/codex/codex-startup-trust.test.ts`. - The runner TypeScript build passes. The full local build stops because the test workspace has no Cargo executable. - The broader Codex suite passed 271 of 272 cases before the source conflict resolution. One unrelated fixture expects the runtime's workspace path to be absent. - The repository type check exited 137 in the DB package. The full local test command was stopped after resource exhaustion. - The final files merge cleanly with master `976cdc77ed564dc3e014621505890b647e554a7f`. `git diff --check` passes. - [CI for the verified head](https://github.com/paperclipai/paperclip/actions/runs/38664681165) passed. All 55 active checks passed; two Storybook checks were skipped. Snyk also passed. [Greptile cycle 3](https://github.com/paperclipai/paperclip/pull/16020#issuecomment-6182103775) scored this head 5/5 with no actionable findings and no open review threads. ## Risks - This update changes the parser used by the runner and server. CI must check both packages. - The trust edit must preserve unrelated TOML values and bytes. Regression tests cover this behavior. - Local resource and tool limits prevent a complete local validation result. The full remote CI run passed at the verified head. ## Model Used OpenAI GPT-6 through Codex helped with the compatibility fix, tests, and review. The runtime does not expose the exact model snapshot or context window size. The agent used repository tools, code execution, and GitHub tools. Dependabot produced the original dependency and lockfile update. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — the focused 28-case suite and strict type check pass; broader limits are listed above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — this PR records the compatibility behavior and verification - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- ## Original Dependabot Update Bumps [smol-toml](https://github.com/squirrelchat/smol-toml) from 1.8.0 to 1.9.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/squirrelchat/smol-toml/releases">smol-toml's releases</a>.</em></p> <blockquote> <h2>v1.9.1</h2> <h2>What's Changed</h2> <ul> <li>fix: reject invalid extra exponent sign by <a href="https://github.com/cyyynthia"><code>@cyyynthia</code></a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1">https://github.com/squirrelchat/smol-toml/compare/v1.9.0...v1.9.1</a></p> <h2>v1.9.0</h2> <p><strong>Huge update</strong>!!! This is most likely the largest update the library received since its release, with lots of new features and improvements.</p> <h3>Performance improvements</h3> <p>Significant parts of the internal parse logic have been rewritten, improving performance by <strong>1.5x-2x</strong>. The library was already comfortably ahead of the others, but it is now faster than ever, sitting at <strong>4x faster parse performance</strong> than the closest maintained implementation.</p> <p>Problematic code paths have also been replaced by safer implementations, solving potential DoS vectors. See GHSA-r4xh-jqrq-34v2.</p> <p>Note: the objects returned by the library now have a <em>null prototype</em>. This is a transparent change for 99.9% of users, and is one of the most significant contributors to the major performance gains in this version.</p> <h3>Full Temporal support</h3> <p>Version 1.8.0 brought support for Temporal in <code>stringify</code>; now the library is also able to emit Temporal objects instead of its own ad-hoc <code>TomlDate</code> object. It is not enabled by default, but it will become the default in v2. Enable by setting <code>useLegacyDate: false</code> in the parser's options.</p> <h4>Better Temporal support in stringify</h4> <p>Temporal support has been improved since it released: Temporal objects that cannot be represented (such as <code>Temporal.PlainMonthDay</code>) now throw an error (instead of silently emitting a bogus object).</p> <p>A new option has been added to <code>stringify</code> to disallow Temporal objects that cannot be fully represented in TOML. This includes <code>ZonedDateTime</code> objects with a IANA timezone attached instead of a plain offset, and dates with a specific calendar value set. Enable by setting <code>strictTemporal: true</code> in the options.</p> <h3>Handling of unsafe keys</h3> <p>Since its release the library has been protected against prototype pollution attacks, setting properties like <code>__proto__</code> using safe mechanisms that do not trigger prototype pollution. However, while the returned objects are safe on their own, they may become problematic if used carelessly.</p> <p>Inspired by <code>secure-json-parse</code>, the library now offers a way to either drop unsafe properties from the returned object, or to throw an error and reject documents altogether. By default, these potentially unsafe keys are preserved and returned.</p> <h3>Miscellaneous updates</h3> <ul> <li>Unicode BOM is now gracefully accepted and ignored.</li> <li>Table array headers are now properly checked again. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/65">#65</a>.</li> <li>Closed certain gaps where invalid whitespace would be accepted. Reported in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/61">#61</a>.</li> <li>Bogus local date and local time values with a UTC offset are no longer accepted.</li> <li>Certain error messages are more accurate and handle errors at line boundaries better.</li> <li>The default export of the lib is now formally deprecated; use a <code>import *</code> instead. Proposed in <a href="https://redirect.github.com/squirrelchat/smol-toml/issues/50">#50</a>.</li> <li>On Node 20+, strings that contain lone surrogates are now normalised to well-formed strings.</li> <li>On Node 20+, keys that contain lone surrogates are now rejected.</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0">https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/squirrelchat/smol-toml/commit/69aa050fef018b71799519bb2cbb54f158a0c5f9"><code>69aa050</code></a> fix: actually properly fix exponent issues</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7be7084d24c85909cec84ecaafe39737ef4e4472"><code>7be7084</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/be3290da3309502447e020ce106bb8a11d99e580"><code>be3290d</code></a> chore: upgrade deps</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/86c76134ad7a50e8e101ef7fd82cb0f155b278d8"><code>86c7613</code></a> fix: reject invalid extra exponent sign</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/6f9739a5ab481f57c3112024bc4947534b580b58"><code>6f9739a</code></a> fix: gate <code>[is|to]WellFormed</code> (Node 18 compat)</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/a73ca32c7c9237f744c880d900ad15307a0cef56"><code>a73ca32</code></a> fix: no Temporal with toml-test when Node < 26</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/7727890604580bd3fc2d7d2f79ad6e67b745ebe7"><code>7727890</code></a> chore: version bump</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/641903d60d47900080602b6a83e919d2a526547f"><code>641903d</code></a> chore: rewrite README.md</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/2df14c57945fec5a3737bc0cdce06a436bf9a77e"><code>2df14c5</code></a> fix(types): make it work if Temporal doesn't exist</li> <li><a href="https://github.com/squirrelchat/smol-toml/commit/3eaa44ef54b859990969935664b12023b26d0eae"><code>3eaa44e</code></a> chore: update benchmark harness</li> <li>Additional commits viewable in <a href="https://github.com/squirrelchat/smol-toml/compare/v1.8.0...v1.9.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Devin Foley <devin@devinfoley.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandClaude Opus 5.5
|
5e8956c207 |
fix(heartbeat): retry legacy runs a restart stopped before dispatch (#16355)
A hard server restart can stop a legacy run after it is claimed but before the adapter is dispatched. The next boot reaped it as failed/process_lost with no retry and held it for operator reconciliation, so the task wake was lost.
The legacy controller only writes executionStage "dispatching" in the lease renewal that admits the adapter. The orphan reaper now revokes the expired foreign-boot controller first, and uses the stage and process identifiers returned by that same UPDATE ... RETURNING. If the run is still "preparing" with no process identifiers, it records bootstrap evidence (providerWorkStarted: false) and uses the existing one-time process-loss retry. In "unexpected" mode, run-failure reporting does not send this retried restart interruption to Sentry. Runs that reached dispatching, native runs, and runs owned by the current boot are unchanged.
Verified exact head
|
||
|
Devin FoleyandPaperclip
|
6fae6f2cb8 |
fix(wake-queue): preserve pending work when a queued run is stopped (#16357)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The wake queue releases task locks and can promote saved follow-up work when a run ends. > - An operator can stop a queued run before dispatch selects its runtime. > - The legacy release path can promote the old follow-up after that Stop. > - For an obsolete routine cycle, this can conflict with the current active cycle after cancellation has already committed. > - This pull request preserves the deferred input without promoting it at that release boundary. > - The benefit is that a queued Stop stays stopped and does not return an error from starting an obsolete follow-up. ## Linked Issues or Issue Description Related: Refs #14300 and Refs #16339 describe other pre-dispatch cancellation paths. Related PR #16238 changes former-owner wake selection. This change addresses expected operator cancellation before legacy runtime selection, without changing those contracts. **What happened?** Stopping an unstarted legacy run committed its cancellation, then tried to promote a deferred wake. When another cycle of the same routine was already active, the issue lock update failed against `issues_open_routine_execution_uq`. The cancel request returned HTTP 500 even though the run was cancelled. Without a competing cycle, the Stop launched the old follow-up. **What did you expect to happen?** Release the stopped run's own locks and preserve deferred input for a later eligible instruction. Do not start the old queue as a side effect of Stop. **Steps to reproduce** Create two cycles of the same routine. Keep the current cycle active. Give the old cycle an unstarted legacy run with a verified bootstrap record and a deferred wake. Cancel that run as a board operator, then release its issue execution. The added database regression reproduces the failed lock update before the fix. **Paperclip version / commit** Reproduced on parent commit `976cdc77ed564dc3e014621505890b647e554a7f`. **Deployment mode** A remote runtime exposed the symptom. The regression uses embedded PostgreSQL and the real wake-queue adapter. ## What Changed - Recognize an expected operator Stop only when the legacy run has no runtime-selection time, start time, process-start time, or process identity. - Use the existing release boundary to preserve deferred input without promoting it. - Add database regressions for ordinary Stop, a competing routine cycle, repeated release, provider cancellation, runtime selection, and dispatched-run reconciliation. - Document the behavior and update both recovery maps. ## Verification - Before the fix, both queued Stop cases failed. The competing-cycle case raised `issues_open_routine_execution_uq`. The three control cases passed. - `node node_modules/vitest/vitest.mjs run --project @paperclipai/server server/src/modules/wake-queue`: 7 files and 214 tests passed, including all five new database cases. - `node scripts/check-maps.mjs` and `git diff --check` passed. - `pnpm -r typecheck`, `pnpm build`, and all 10 map-script tests passed locally. The complete CI suite is green on `8e204eef87f1a593df5c183abef764e4b9eb5fd1`. The remaining serial local full-suite run was stopped once that unchanged-head CI verification passed; a local full-suite pass is not claimed. Apex review passed at 5/5 on the same head with no review threads. ## Risks - The change is limited to recorded operator cancellation before legacy runtime selection. Selected runtimes, dispatched work, provider cancellation, and native stop acknowledgement keep their existing paths. - Deferred work is retained. A later eligible instruction must still use normal admission and all current gates. - No schema migration or new authority is required. This regression is not proof of a live provider stop. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution. The exact serving snapshot ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
976cdc77ed |
fix(recovery): expose settled holds and their preserved decisions (#16169)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task recovery preserves work when execution or tool outcomes are uncertain. > - Automatic recovery can settle its record while keeping replay blocked. > - The recovery-actions API omitted those records unless they held workspace files. > - Operators and agent tools could therefore see an empty recovery history for a task that still had an execution hold. > - This pull request lists those holds and shows their matching decision in the existing stopped-run notice. > - Existing stop verification, continuation, and file repair gates remain in force. ## Linked Issues or Issue Description **What happened?** A cancelled or failed task can retain a settled recovery record with `automaticRecovery.replay: blocked`. The task remains held, but the recovery-actions API returns no action. The board notice also lacks that record's decision. **Expected behavior** List settled execution holds that still block replay. Let an authorized board user inspect the current action and run in Recovery details. Preserve the existing guarded continuation and required-file repair paths. **Steps to reproduce** 1. Stop a run whose outcome requires inspection. 2. Let automatic recovery settle its record with replay blocked. 3. Read the task's recovery actions and open its stopped-run notice. 4. Before this change, the action list is empty unless workspace recovery is required. **Paperclip version or commit** This PR includes current master through `66424362a85`. **Deployment mode** Authenticated and local deployments with persisted execution recovery holds. The original API correction and route regression are by Tao Xie (@TaoXieSZ). This update preserves those commits and adds the board inspection path, current documentation, and regression coverage. Related: #14840 describes controller lease loss. This change exposes recovery records; it does not change controller lease renewal. ## What Changed - Include settled execution reconciliation holds with replay blocked in the company-scoped recovery-actions response. - Read the matching action from the existing stopped-run notice. Show its decision under Recovery details, including when active recovery is empty. - Reject mismatched historical records in the notice. Bind history to the current action/run and refresh visible decisions every five seconds. Keep history read errors visible beside the current guidance. - Preserve existing run inspection, new-message continuation, and required-file repair controls. No new reconciliation form is added. - Update the execution documentation and both recovery capability maps. ## Verification - Recovery route suite with real embedded PostgreSQL: 80 passed, none skipped. - Notice and retry-client suites: 36 passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - `pnpm check:token-gates`: passed. - `node scripts/check-maps.mjs`: passed. This checks structure only. - `git diff --check`: passed. - Full `pnpm test:run` was started. It is not yet a full local pass. Initial database tests were affected by missing packaged macOS library aliases; the aliases were restored in this worktree's dependencies, and the complete recovery route suite then passed. - All CI checks passed on `09e4461e4d3a0960a4dc8bc270a2a10c628bf524`, including the full test matrix, typecheck, build and browser checks. Optional Storybook jobs were skipped. - Apex: 5/5 on the same head. Both review threads are resolved; the reviewer confirmed the existing guarded continuation design. ## Risks - The API response now includes settled records that still block replay. Consumers must distinguish a settled record from permission to continue. - Recovery history is supplemental. A failed history read keeps the current blocker and guidance visible. - The UI binds the decision to both the current action and run. It does not certify tool outcomes or grant execution authority. - No schema, dependency, telemetry, or runtime admission changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and code execution, assisted with the board inspection and verification follow-up. The exact serving snapshot ID and context window size are not exposed in this session. The original API work is retained from Tao Xie's contribution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted suites; full-suite limits above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
66424362a8 |
perf(runtime): skip discovery without a Browser Use connection (#16345)
Skip full tool discovery when no enabled, active Browser Use connection exists for the run company. Configured connections retain existing authorization and discovery.
Verified exact head
|
||
|
DottaandPaperclip
|
c41e00bca4 |
fix(server): index workspace source task enumeration (#16342)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent startup checks access to every source retained by a workspace. > - Repository admission repeats these checks while it holds a company lock. > - PR #16334 indexed the workspace privacy query. The next source-task query still cast the task primary key to text. > - That second query scanned company history for every workspace and delayed other agent launches. > - This pull request keeps the task column indexed and guards the retained identifiers before conversion. > - The benefit is faster startup with the same source validation and access decisions. ## Linked Issues or Issue Description Refs #16334. **What happened?** Agent launches still spent minutes preparing repositories after the first authorization query fix. The source-task lookup scanned 14,191 task rows for one matching source. **Expected behavior** Source validation should use the task primary key and workspace-binding index. It must still reject missing, malformed, noncanonical, and foreign-company sources. **Steps to reproduce** 1. Seed company task history and a workspace with attached and retained source tasks. 2. Call `assertTaskWorkspaceAccess` and capture its source-task SQL. 3. Run `EXPLAIN (ANALYZE, FORMAT JSON)` on that query. The baseline scans unrelated task history. ## What Changed - Convert retained text identifiers with the existing canonical UUID guard. Keep the indexed task column as a UUID. - Exercise the real source-access service against PostgreSQL. Verify complete source results, malformed and uppercase input behavior, and rows examined by the plan. - Document the second source-validation query and its regression coverage. ## Verification - Red: the new regression failed because the source lookup examined 1,202 rows for two sources. - Green: four focused workspace suites passed, 147 tests total. The final regression also checks malformed and uppercase query inputs. - Live read-only comparison: two warmups, then five samples per variant in alternating order. Median client query time changed from 4.88 ms to 0.35 ms. Complete result rows matched. Shared buffer hits changed from 3,984 to 5. These measurements cover this SQL statement, not total agent startup. - The complete source-access helper also passed for the same retained reader in read-only transactions. Its median changed from 15.75 ms to 10.75 ms; repository admission still repeats other checks. - `pnpm -r typecheck` and `pnpm build` passed locally. The staged Linux build and all seven final regression cases passed on Node 24.20.0. - All 55 current-head checks/statuses passed; two Storybook checks were skipped. Greptile reviewed `6cfb755` at 5/5 with no findings. - The full local test run encountered a tracing-test failure in `native-session-executor.test.ts`. Both that test alone and the complete file passed on rerun (698 tests). The remaining full-suite run is still in progress. ## Risks - This changes SQL planning on an authorization path. The canonical guard preserves the former exact lowercase text match and avoids UUID parse errors. - Company filters, missing-source checks, project checks, and per-source access decisions remain in place. - No schema migration or new index is required. Repeated checks across large repository histories remain a separate source of startup work. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, shell tools, code execution, and browser verification. The exact model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |