mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 06:42:06 +01:00
100b364264c225085fb6633f11bde002f9afde19
6044
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
100b364264 |
Preserve selected Google services when setup resumes (#18048)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections control which external services an agent can use. > - Google Workspace setup lets a person choose services and read or write access. > - A draft lost that choice after a declined or expired OAuth handoff. > - Finish setup could then show all services with write access instead of the saved narrow choice. > - This pull request saves the exact draft choice without treating it as granted access. > - The benefit is safe setup recovery that preserves the person's choice and existing account limits. ## Linked Issues or Issue Description Refs #15545. The duplicate search found no open PR for Workspace draft consent recovery. Related #18000 concerns satisfied gateway access requests, not saved OAuth setup intent. **What happened?** A cold Workspace draft resumed with the new-account service defaults. The selected request lived only in short-lived OAuth state. Denial consumed that state. The picker ignored draft connections. **Expected behavior** Finish setup restores the same person's saved service choice. It does not use another person's choice, a broad profile hint, or a credential grant as draft intent. An old draft with no reliable choice asks the person to choose services. **Steps to reproduce** 1. Start a new Google Workspace connection. Select Gmail read-only. Turn off Drive and Calendar. 2. Continue to Google, then decline authorization. 3. Close the setup page. Return through Finish setup. 4. Open the service picker. Before this fix, all three services can be enabled with write access. **Paperclip version or commit** Reproduced in tests on master `820c458465fa40137c81fe510f3e6dc573b4f895`. **Deployment mode** Authenticated and local source builds. The source regression uses real PostgreSQL fixtures and the production React setup components. Live provider acceptance of this repair is still pending deployment. ## What Changed - Save validated, non-secret draft intent before managed or customer-owned OAuth. Bind it to company, connection, actor, grant kind, and subject. - Restore that intent after denial, state expiry, and cold Finish setup. Keep active reconnect bound to its saved grant. Do not change new-account defaults. - Preserve agent-to-human setup handoffs. Consume only the addressed person's reviewed proposal. Keep other people's proposals. - Lock and merge the current draft before saving intent. Return 409 if its credential identity or method changed during setup. - Preserve the latest saved choice across overlapping customer-owned starts. Merge both current configuration records under short locks. Reject stale client-binding changes. Keep each OAuth state's own requested scopes. - Add shared, service, handoff, picker, and cold-page regression tests. Update the connector contract and existing feature plan. - Route the default ungrouped test/release paths through six sequential exact-cover chat partitions. Preserve remaining server/native coverage, explicit grouped CI, selection-first execution and shared-database paths. - Correct the retention test driver so pending owned real I/O does not advance logical deadlines. Pump zero-duration connection timers and add a held-I/O regression; keep production scheduling and original timing assertions unchanged. The shared helper is needed by the server and picker to validate the same narrow binding. It does not resolve credentials or grant access. No production schema migration, new route, provider scope or deployment configuration is added. The test-harness changes do not change production timers. ## Verification ### Current candidate: source qualification passed; live acceptance pending - Current published candidate: `82d13d4d698a5579378b5d506cb7a1f9cb64b5c8`, tree `78d6a821ce53f481fe3e9991edce56f0acffb22f`. It combines the Workspace repair with launcher commit `bf29c9` and retention-test commit `e25987`, then integrates master `493e229d93ba7798e80152f4549e2e9177ce50ae`. The integration brings 40 upstream commits across 713 files. One launcher conflict keeps upstream validated CI selection and its selection-first path, then applies six default chat partitions only without a selection. The authored PR diff has 16 files, 833 additions and 100 deletions; it adds no authored lockfile or `.github` changes. The public head is confirmed at this revision. Fresh CI and review passed as recorded below. Source qualification is complete using exact-tree full CI and the local results below. The separate local whole command was interrupted by host disk protection and is not called a pass. Live repaired-draft acceptance remains pending deployment. - Exact clean `82d13d4` passed one Node 24.19.0 builtin-only run: 41/41 cases (37 launcher and four CI-workflow cases), zero failures/skips. HEAD, tree and relevant file hashes stayed unchanged and clean before, at start and after. This proves launcher fixtures, not real chat execution, current typecheck/build, the dependency-backed selection graph, or full feature acceptance. - The new default ungrouped whole command uses six sequential exact-cover chat partitions, then the remaining server suites, workspaces and serialized suites: 28 expected invocations. It preserves all native suites, including upstream usage coverage. The plain ungrouped release launch path also changes. Explicit grouped CI, validated selection and shared-database execution stay unchanged. Launcher hard RED had three expected body failures and two unchanged passes; the repaired focused five and final 37 passed. A later 41-case staged-merge pass is retained separately; the clean 82 run above is the frozen proof. - Fresh [CI run 41153113944](https://github.com/paperclipai/paperclip/actions/runs/41153113944) completed successfully on October 31 at 01:14:53 UTC. Current head has 52 successful check runs, four conditional skips and no pending/failing checks; legacy Snyk also passed. The skips are test-file selection, the optional paired benchmark and two Storybook checks. The full server/chat/workspace/serialized matrix, all eight e2e shards, normal build, build-gap typecheck, Runner checks and final aggregate gates passed. Build and all three chat job logs identify checkout `e5e831fd3a8534d0d0f1e892118c0f8fbe12dd92`, merging this head into `493e229d93`. Its Git tree is exactly `78d6a821ce53f481fe3e9991edce56f0acffb22f`, identical to the clean candidate. The six real chat partitions passed 314, 358, 273, 239, 321 and 311 cases: 1,816 total. Other cases shown as skipped in each partition were selection exclusions, not extra passed cases. This is current tree-equivalent CI proof, not a pass of the separate default ungrouped local whole command. - Fresh [Greptile summary](https://github.com/paperclipai/paperclip/pull/18048#issuecomment-6473856992), edited October 31 at 01:05:04 UTC, explicitly reviews exact head `82d13d4` at 5/5 with no actionable code issue. Its exact-head check passed. The prior overlapping-start discussion is resolved. This review is source-only, not provider or live acceptance. - Exact clean `82d13d4` passed full recursive typecheck and normal `pnpm build` on the independent darwin/arm64 snapshot. Typecheck finished on October 31 at 01:27:33 UTC; build finished at 01:28:33 UTC. Both exited 0. Normal native preparation and code signing passed. Source HEAD, tree and tracked/untracked state stayed unchanged and clean. `pnpm check:token-gates` also passed all token, palette and ClipLab checks on this candidate. The earlier forced-clone `ENOSYS` preflight remains recorded as setup failure, not an assertion result. - The unchanged default `pnpm test:run` started at 01:51:33.796 UTC on clean exact `82d13d4`. All six real chat partitions passed: 358, 314, 273, 239, 311 and 321 cases, or 1,816 actual cases. The original FIFO and ambiguous-retry cases both passed under their unchanged 15-second limits. General-server then reported 37 completed file rows with 5,438 passed cases and five default remote-continuity skips, including all 498 tool-access service cases and all 83 connection-route cases. At 02:39:44 UTC, the unchanged 5-GiB host-disk guard terminated the owned run. The child ended with SIGTERM and the container/harness exited 143, with no OOM or recorded assertion failure. Only seven of 28 RUN invocations started. Retention had no completed local row. Later invocations did not run. This whole local command is interrupted/incomplete, not passed. - A coverage check found no default suite missing from current full PR CI. The maintained launcher tests prove exact coverage for all 12 PR server shards, chat and five native suites; all configured workspace projects; and all nine serialized shards. The final Runner CI shard executes the five native server suites. The 41 local launcher/workflow cases cover six sequential chat children, per-child isolation, native retention, fail-fast propagation and unchanged explicit selection/shared-database paths. The actual six-chat run above exercises that changed default path. The remaining unqualified local difference is uninterrupted sequential whole-command composition and resource accumulation, not omitted test selection. Current exact-tree full CI supplies broad source-test qualification; no local whole-pass claim or provider acceptance is inferred. - A task-owned remote runner was considered to complete that separate composition check. Its provider rejected the requested allocation with per-sandbox limits of 8 GiB RAM and 10 GiB disk. No sandbox was admitted and no remote tests ran. No tests, timeouts, assertions, database synchronization, source or provider settings were changed to avoid the capacity limit. Historical a0 failures remain below. No deployment or live repaired-draft acceptance has run. ### Bounded follow-up on the prior source - On a0, one finite phase-traced chat diagnostic executed exactly the two failed cases. Both passed (1,374 and 1,767 ms); 1,814 other cases were filtered, not passed. This did not reproduce a blocked await or establish the original monolithic timeout cause. No assertion or 15-second limit changed. - The retention diagnosis preserved an initial zero-body selector failure, then corrected only the selector: one original case passed in 783 ms, with 42 filtered. Trace showed the old settling helper advancing logical time while real I/O was active. A new held-I/O regression then failed hard because the old helper advanced 1,000 ms. This proves a test-driver defect, not the precise cause of the old 5,340 ms full-run gap. - The first manual-clock repair failed the complete 44-case file: 43 passed, one body hit 15 seconds and cleanup hit 30 seconds. Its planned mutant did not run. Source then identified a fresh PostgreSQL connection's zero-duration timer. The corrected driver pumps zero-duration timers while initial readiness or tracked I/O is pending, without advancing logical Date. That pre-integration file passed 44/44 with no skips. A separate scratch-only 5,000 ms normal-continuation mutant failed the original hard `5000 < 3000` assertion; its expected failure is not a test pass. The original page-count, fairness, expiry and quiet assertions stayed intact. The authored repair does not change the production worker. The merged master changes an older retention case and dependencies; these pre-integration results are not a complete 82 retention or whole-suite pass. ### Historical qualification: SHA-bound, not 82 acceptance - Historical a0 candidate: `a0f9089fc1ff466bf4e1eeddc5b1a910d09e380b`, tree `b9d73713319dcd52b9eaa6053008b5190e43fdf1`. It adds the overlapping-start repair to parent `6c18`, which already merged pinned master `1511c92965325249ed6b52dd5be52eb6d736fbfa`. That candidate changed 12 files. Its CI and review passed; its local full qualification remained failed/incomplete. - Greptile's previous `6c18` review found P1 [Latest service choice gets lost](https://github.com/paperclipai/paperclip/pull/18048#discussion_r4425965828), confidence 4/5. Two hard-assert regressions reproduced it on untouched `6c18`: the already-bound path overwrote the later choice, and real first-use client stamping also restored the old choice. Existing OAuth states/grants were not widened, but a later setup could restore an older request. The new repair covers both write boundaries without holding locks over external work or copying another attempt's scopes. - The complete affected service file passed 498 tests with no failures or skips. All 31 required controls ran, including both unchanged RED cases, 14 new client-drift cases, seven earlier identity guards, reconnect/cold-resume cases and upstream transaction tests. The tested three-file patch is byte-identical to this commit; the connector contract update is separate. Independent scoped review found no additional actionable issue. Fresh Greptile reviewed exact `a0f` at 5/5 with no actionable findings. The previous P1 thread is resolved. - On exact `a0f`, the five shared/UI files passed 312 tests. An initial complete connection-management run then failed: 74 passed, eight timed out at the unchanged 15-second limit, and one AI-account verification returned 422 instead of 201. The Google addressed-user handoff passed. The container exited 1 without OOM and stopped before chat/token/whole tests. This material failure remains retained and its cause unclassified; the later controlled recovery below is not a causal explanation. - One diagnostic executed the first failed gateway case and the two AI cases with temporary phase-only tracing. All three passed at 8.516, 5.511 and 3.523 seconds; 80 other cases were filtered. Original hard assertions and timeouts stayed unchanged. No timeout or suspected late provider cleanup reproduced, so the complete-file failure remains unresolved. Vitest exited 0; the harness exited 1 because its trace parser missed ANSI prefixes. All 203 events were recovered from retained logs without rerunning tests. Instrumentation was removed. This is not acceptance. - Historical a0 CI [41097596364](https://github.com/paperclipai/paperclip/actions/runs/41097596364) completed successfully: 52 successful checks, two expected Storybook skips and no failed/pending checks. Legacy Snyk passed. All 48 CI jobs passed, including all 1,816 chat cases and explicit passing lines for the original 20 local failures. [Server 11/12](https://github.com/paperclipai/paperclip/actions/runs/41097596364/job/124207469276) explicitly passed the complete 83-case connection file in 129.49 seconds, including all nine local failure titles. The actual checkout was synthetic merge `06851da6eb969d63e1c7dc9f7c95f08d09fa4ff1`, combining `a0f` with `aa58fa107389e8b9ae5949f7ad815943dfd94292`. This is not standalone local `a0f` qualification and does not erase the nine local route failures. - Linux admission uses Node 24.22.2, pnpm 9.15.4 and Rust 1.97.1. Frozen offline install, exact native hashes and real migrated PostgreSQL 18.1 passed. Native/protocol/build/lock inputs are identical between `6c18` and `a0f`; validated cache reuse retains the actual `6c18` native-build provenance. The selected packaged release binary is not called a fresh `a0f` build. Database synchronization settings stayed on. This admission alone does not explain the route failures. - A fresh Mac snapshot of exact `a0f` passed frozen install, workspace prerequisites, full sequential recursive typecheck and normal `pnpm build`, both on their first attempt. Typecheck completed at 17:43:05 UTC, build at 17:44:43 UTC; both exited 0 with clean tracked/untracked source. Pinned Node 24.19, pnpm 9.15.4, Rust 1.97.1, package concurrency one, Cargo jobs one and a 12 GiB V8 heap ceiling were retained. Largest-process maximum RSS was 7,668,154,368 bytes for typecheck and 10,038,837,248 bytes for build, not aggregate tree peaks. The first build correctly withheld its freshness stamp because 125 ignored, pinned Copilot assets appeared during preparation; exact path/hash analysis explained this. The normal build then recorded a genuine stamp and reused Cargo outputs in 1.18 seconds. No manual stamp, source workaround or gate retry. This is darwin/arm64 static/build proof, not Linux packaging/runtime or full tests. - The historical a0 candidate's token checker passed all four gates across 1,410 files, run directly with pinned Node 24 against the script behind `pnpm check:token-gates`. - The single controlled complete-file comparison passed: baseline `6c18` 83/83 in 287.07 seconds, then candidate `a0f` 83/83 in 293.03 seconds, with zero skips. All nine earlier failure titles passed on both. Both child/container exits were 0, no OOM/interruption; source and native/resolver/PostgreSQL guards matched. This ran only after all owned Mac compilation ended, with fresh processes/databases and identical four-CPU-quota/6-GiB/overlay inputs. No filtering, instrumentation, assertion/timeout/source changes or retry loop. This establishes standalone complete-file recovery, not the cause of the earlier eight timeouts/AI 422. - The first unchanged full `pnpm test:run` on exact clean `a0f` ran from October 30 at 18:01:43 UTC to 21:06:19 UTC. It failed naturally with test and container exit 1, without OOM. General-server reported 1,257 passed, one failed and seven skipped files; 28,633 passed, one failed and 103 skipped tests. Only the first of 22 invocations ran. The later four groups and 21 invocations did not run. The sole failure was the unchanged chat-shard CLI meta-test: its nested projectless pnpm lookup hit the offline network guard. This was a failed full gate, not Google provider acceptance. - A finite environment pair reproduced that one meta-test failure, then passed the same unchanged case after supported `corepack prepare pnpm@9.15.4 --activate` as the existing non-root node user. The cached pnpm payload was identical. Only the missing node-user last-known-good record changed. Core source, dependencies, lockfile, fixtures, assertions, timers and offline guards stayed unchanged. This isolates the nested collection defect; it does not explain the earlier route failures or complete the full gate. - The corrected unchanged full command ran from October 30 at 21:29:03.500 UTC to October 31 at 00:02:44.207 UTC on the same exact clean `a0f`. It failed naturally with exit 1, signal null and no OOM. Source stayed clean before, at start and after. It retained two visible CPUs/quota, 6 GiB, private IPC, normal overlay, matching native inputs, synchronized real PostgreSQL 18.1, and the default five groups/22 invocations. Only the first general-server invocation ran; the other 21 did not run. General-server reported 1,265 files: 1,256 passed, two failed and seven skipped. It reported 28,737 tests: 28,631 passed, three failed and 103 skipped. The two chat cases emitted `Test timed out in 15000ms` at `chat-channels.integration.test.ts:77310` and `:81790`, with durations of 15,045 and 15,058 ms. These were the durable pre-run FIFO notice and ambiguous GitHub text restart/retry cases. The retention case emitted `expected 5340 to be less than 3000` at `skill-version-retention.test.ts:593`. Its real duration was 1,727 ms; 5,340 ms was the recorded fake-time directory-page gap, not a deadline timeout. The timeout stacks do not identify the blocked internal await. The cause of these failures remains unclassified. Inside this same failed run, the complete 83-case connection file and 498-case service file passed. The former Corepack meta-case also passed in 6,396 ms. These component passes do not waive the failed/incomplete full gate. No rerun, cancellation, timeout/assertion change or full-pass claim. A later fresh two-case phase trace passed both chat cases; it did not reproduce or establish the original timeout cause. At that checkpoint, the PR stayed draft. - Prior `6c18` targeted checks passed 930 tests across eight files and token gates. Its selected queue group passed all 53 cases, including the original 20 failures and 12 new named controls. Backlog cases took 10.49 and 8.45 seconds under the unchanged 15-second limit. Another 1,763 chat cases were intentionally filtered. This is historical targeted recovery, not a full pass or proof of the new repair. No timeout or assertion was relaxed. - Prior `6c18` full local qualification failed during serial typecheck: server TypeScript was killed with code 137. Docker confirmed OOM in the 6 GiB container; full tests and build never started. The failure is retained. The new Mac resource plan avoids repeating that undersized run and does not change the shared Docker VM. - Prior CI [41089793558](https://github.com/paperclipai/paperclip/actions/runs/41089793558) completed successfully for synthetic merge `a73bffcfa3ede051961bcabcfa576c703d10c932` (`6c18` plus `abc270ee4c03e13b4d6eaad156a41b4ebed4bb0a`). All 48 CI jobs passed; all 1,816 chat cases passed, including explicit log lines for the original 20. Check runs were 51 success, two expected skips and one Greptile failure; legacy Snyk passed. These results do not qualify `a0f` or erase its local route failures. - Red proof on the unmodified master baseline: cold picker and Finish setup restore broad defaults; managed and customer-owned drafts lack saved intent. The stale setup tests also fail before the lock guard. - Live staging baseline `b25f3e2af8e77d882cd22d36ca55640e6468e46f` also reproduces the ordinary failure: select Gmail read-only only, cancel at Google, reload the connector catalog, then Finish setup. All three groups return enabled at read/write. Authorization was declined; no new Google access was granted. This is baseline failure evidence, not a deployed-fix pass. - Historical affected suites: 800 tests passed before the final lock guard. The final focused service and handoff cases passed 126 tests after that guard. These are not results for the integrated candidate. - Historical `pnpm -r typecheck` and final lock-guard server typecheck passed. The historical a0 candidate passed the full recursive gate separately, as recorded above. - `pnpm build`: passed again on the final lock-guard source in commit `1c31cde0ed199fbf4e41b7649590554f7886dfc2`. - Historical `pnpm check:token-gates` passed; the historical a0 candidate's direct checker pass is recorded above. - `pnpm test:run` on macOS failed on the frozen commit. It exited 1 after 9,432.41 seconds. General server results: 1,195 files passed, 10 failed, eight skipped; 26,706 tests passed, one timed out, 157 skipped. Nine setup hooks timed out while starting embedded PostgreSQL. The retained log records host-wide semaphore exhaustion (`semget`), not disk exhaustion. One unchanged publication-text performance case also timed out. Its cause is not established. Later workspace and serialized groups did not run. This is a failed and incomplete local gate. - The independent full-source Linux run also failed and was incomplete on the same frozen commit. Native and real-database admission passed. Chat integration reported 20 failures in 1,593 tests. Tool-access passed all 479 tests. Native-session-executor passed all 965 tests. Docker interrupted the run after about 48 minutes on October 29. The container exited 143 and was not OOM-killed. The run produced no final group totals or detailed assertion summary. Shell trap metadata reports zero, but this is not a valid test-pass result. Source and evidence remain unchanged. The focused diagnosis below retains those failures; their cause is not assumed from the interruption. Neither historical local full run passed. - A fresh focused replay on the frozen source then executed exactly those 20 case bodies. All 20 failed; this was not a setup-hook failure. The two backlog cases reached the unchanged 15-second timeout. The 18 FIFO cases failed on missing notices or receipt rows. All 18 FIFO cases passed when run alone in a fresh process and database. This supports an order interaction, not a complete cause or baseline conclusion. A separate two-backlog control used RAM-backed temporary storage and real PostgreSQL with synchronization settings on. One backlog case passed in 10.65 seconds; the other still timed out at 15 seconds. No source, assertion or timeout was changed. The full local gate remains open. - Earlier mixed-revision run: it began before the final guard and imported older source before collecting later regression cases. It failed those new cases with that older cached module. This run is retained, not counted as candidate acceptance. The fresh focused run passed all seven stale-setup cases. No case was skipped to hide a failure. - Historical remote gates for `1c31` were terminal and clean: 53 successful contexts and two expected Storybook skips. Main CI [40720536514](https://github.com/paperclipai/paperclip/actions/runs/40720536514) passed. Greptile reviewed exact head `1c31cde0ed199fbf4e41b7649590554f7886dfc2` at 5/5 with no actionable findings; the separate review job also passed, and there were no inline review comments. These results do not qualify `6c18`. - CI source qualification: the chat jobs checked out synthetic merge `ef36746b3d2ec52ba5727dcbe4da265a03421fe4`, which merges the PR into `7d75ae0569e41786640cf09647ae8849411d2f22`. All 20 locally failing case names have explicit passing CI log lines. These passes do not establish standalone local acceptance of `1c31`. The merge adds two upstream Teams tests, so it collects 1,595 chat cases instead of 1,593. Neither backlog/FIFO assertions nor their timeouts changed. - Independent source review found an earlier credential-identity race. The guard and seven deterministic cases resolve that finding. A later integration review missed the separate overlapping-start race. Both RED failures and that review miss are retained. The narrow follow-up repair passed 498 service cases, independent source review and fresh remote CI/5-of-5 review; full local qualification remains incomplete. - Source qualification is ready for merge. Live repaired-draft recovery, delivery and Google approval are not established by these results. The local whole command remains explicitly incomplete. ## Risks - Historical drafts without a matching intent now need an explicit service choice. This is deliberate and prevents silent broadening. - A concurrent credential or method edit now returns 409. Reloading lets the person review the changed setup. - Overlapping starts keep separate requested scope sets. Updating draft intent does not relabel an earlier OAuth state's consent request. - Saved intent is not consent. Provider-reported scopes and current action policies still control discovery and dispatch. - No migration is needed. No existing token or grant is rewritten by saving draft intent. - Default ungrouped launch paths now execute six chat children in sequence and 28 whole-command invocations. Exact coverage, local launcher contracts and all six real default chat partitions passed. A complete uninterrupted local whole-command run remains unqualified because of host capacity. - The retention test-clock repair depends on owned-I/O and zero-duration connection-timer progress. The controlled regression, sensitivity mutant and current exact-tree CI protect the original assertion. It did not reach a completed row in the interrupted local whole run. ## Model Used OpenAI Codex, described by the runtime as GPT-5 based. The exact runtime model ID and context-window size are not exposed to this task. Used reasoning, code execution, browser verification, and independent delegated review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including the runtime metadata limit - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the bug report template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance identifier - [x] I have run tests locally and they pass — exact clean 82 passed 41 launcher/workflow cases and all 1,816 real default chat cases, full recursive typecheck, normal build and token gates. Full exact-tree PR CI passed. The separate local whole `pnpm test:run` was interrupted by disk protection and remains incomplete; it is not claimed as a local whole pass. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green — current exact-head checks passed; the verified CI checkout tree equals clean 82. The separate interrupted local whole command is disclosed above. - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — fresh exact-82 summary and check passed; the earlier finding is resolved. This is source review, not acceptance. - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b866fa4bdc |
feat(runner): add pinned Kimi Code native runtime (#18028)
Add the pinned Kimi Code runtime, native instruction and model admission, scoped credentials, and pending-candidate recovery contracts. Retain current master release attestations and provider-pack compatibility. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
909e29dd07 |
fix(ui): show agent-created subtasks inline in the task feed (#18320)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The task feed shows agent messages, documents, and artifacts. > - An agent can create a subtask or a separate follow-up task during that work. > - The reader must open the Tasks tab to find the new task. > - This pull request adds a linked task card to the source feed at creation time. > - The card shows the current title, task number, and status. > - The reader can inspect delegated work without searching a second panel. ## Linked Issues or Issue Description **What existing behavior does this improve?** Task creation visibility in the source task feed. Related work: #13823 added ancestor navigation in the Tasks panel. **Current behavior** Agent-created child tasks appear in the Tasks tab. The main feed has no persistent link card for those tasks. **Proposed behavior** Show each agent-created direct child or source-linked follow-up task once in the feed. Keep the card at creation time when status changes. Keep existing chat handoff and accepted suggestion cards without duplicates. **Reason and benefit** Readers can see which work an agent created and open that task from the conversation. **Breaking changes** None. This change uses existing authorized task reads. It does not add API or database fields. ## What Changed - Pass the existing child and source-task reads from task detail to the feed. - Add a task card with a link, wrapping title, task number, and status badge. Use existing design tokens. - Suppress repeated cards and tasks already shown by accepted suggestions or chat handoffs. - Show task-read failures with Retry. Keep the conversation readable during those failures. - Add feed and page regression cases plus desktop, multiple-task, dark, and mobile stories. Update the product description and delegation recipe. - Keep task selection and rendering in the task feed module. Existing issue APIs continue to own access checks and company scope. ## Verification - Passed: 224 focused tests with `pnpm exec vitest run ui/src/components/TaskChatThread.task-creation-order.test.tsx ui/src/pages/IssueDetail.test.tsx ui/src/components/task-detail/TaskDetailTasksPanel.test.tsx --maxWorkers=1`. - Passed: `pnpm --filter @paperclipai/ui typecheck`. - Passed: `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `node scripts/storybook-catalog.mjs`. - Cases cover creation order, current status, links, company filtering, direct children versus grandchildren, duplicate suppression, and read failure with retry. - Passed: browser render and link assertions for desktop, multiple tasks, dark theme, and mobile at 390px. All four story play phases finished. No page errors or horizontal overflow occurred. Screenshots use fixture task data and the production task feed. They do not prove server persistence or agent execution. - Passed: `pnpm --filter @paperclipai/ui build` after the concurrent memory failure. - The mobile preview frame needed spacing to contain the feed scrollbar gutter. Fixed the frame and repeated the browser checks. - Addressed the first Greptile finding: source-linked follow-ups now load with Streamlined UI off. A regression test covers that setting. - Full local build and typecheck cannot complete because Cargo is absent. The first concurrent UI build and UI typecheck were killed by the workspace memory limit. Both UI commands passed when run separately. The full local test run was stopped to fit that limit. PR CI must supply the full-suite verdict. ## Risks - Low risk. Delayed task reads can insert an older card into a loaded feed. Stable task IDs preserve the card during status updates. - New cards use the same access-checked task data as the Tasks panel. No task mutation or permission change is added. ## Model Used - OpenAI Codex, GPT-6. The runtime does not expose a more specific model ID or context size. Used reasoning, tool calls, code edits, tests, and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d06c5e0eff |
fix(evals): preserve opaque TestOps parameters (#18394)
## Thinking Path > - Paperclip verifies real agent work through Product E2E tests. > - The fixed Chat feature workflow uses TestOps to select and report one case. > - TestOps exposes job parameters as ID/value pairs. > - The schema does not establish that those IDs identify global environment variables. > - The reader used that unsupported relationship to infer parameter names. > - This pull request keeps IDs opaque, preserves the strict allowed values, and records closed read diagnostics. ## Linked Issues or Issue Description Refs: #18322, #18375, #18385. **What happened?** A zero-model bootstrap returned HTTP 404 during context readback. The exact failed GET was not retained and remains unknown. Schema review also found no established relationship between job-parameter IDs and global environment-variable IDs, although the reader queried that namespace to label parameters. **Expected behavior** Read only the existing job-run, launch, and job records. Preserve the job's exact permitted ID/value vector without inventing parameter names. Record which fixed GET class failed, its positive resource ID, state, and available HTTP status without raw responses or URLs. **Steps to reproduce** Use the actual API helpers with a job-parameter ID that has no global environment-variable definition. The new pure controls reject an unexpected definition lookup and verify the opaque vector and diagnostics without a backend or model call. **Paperclip version or commit** This change starts from `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` after #18385. **Deployment mode** GitHub Actions metadata setup in the protected TestOps reporting environment. ## What Changed - Remove the unsupported global environment-variable lookup from job-parameter handling. - Preserve an exact positive ID and `refs/heads/master` value. Use the fixed workflow's existing `evaluate` default. - Treat an operation parameter as such only when its ID matches the protected configured ID and its value is allowed. Keep rejecting empty, duplicate, conflicting, or unknown vectors before writes. - Project only an opaque ID with a fixed-default-ref kind and value, or an unresolved kind with the value withheld. - Add closed diagnostics for the existing job-run, launch, and job GETs. A successful call keeps HTTP status null because the API abstraction does not expose the status. - Sync one reporter-test assertion and document opaque IDs. Reporter runtime and workflow source are unchanged. ## Verification - Four new actual-helper controls fail on the parent. - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs tests/runner-e2e/testops-chat-reporter.test.mjs`: 58 affected pure/mock/FS controls pass on Node 26.4.0. - Three MJS syntax checks and `git diff --check` pass. TypeScript noEmit was not run for this MJS/documentation change. - Two intermediate exact-projection assertion failures were retained before the assertions were synced to the new diagnostic contract. - [Normal Node 24 CI](https://github.com/paperclipai/paperclip/actions/runs/41137009579) passes on exact head `3c256fb8f7559093c5fb063170e26c01b6e8a0a4`: 48 jobs passed and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed this exact head at 5/5 with no actionable findings or unresolved threads. At this verification cut, the optional Contributor trust check was still in progress; it is not an effective required status context. - No actual CLI, API, provider, model, app, database, build, install, or additional bootstrap was run to validate this change locally. ## Risks The original HTTP 404 route remains unknown. Removing an unsupported schema assumption does not prove that it caused that failure. The live job mapping and dispatch remain unqualified. The strict source, actor, selection, run-number, nonce, once-only send, private custody, case, model, grader, and clock checks are unchanged. Unknown metadata still stops. This PR adds no endpoint, workflow mode, permission, credential path, or passing result. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8ff812cdde |
feat(chat): deliver proactive task messages to agent chats (#18374)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Agent Chat gives each person a persistent conversation with each
agent.
> - Tasks and routines produce results outside that conversation.
> - Those runs need a safe way to bring a requested result back to the
person.
> - The server must select the agent's own chat and check the source
task's access rules.
> - This pull request adds proactive chat messages and unread
indicators.
> - People can see the result, open its source task, and reply in the
same chat.
## Linked Issues or Issue Description
**Subsystem affected**
Server, native runner tools, shared contracts, database, and UI.
**Problem or motivation**
An agent can report progress on a task or comment on an authorized task.
It cannot send a routine result to its own standing chat with the
responsible person. A morning review routine therefore cannot leave a
visible digest in that person's chat.
**Proposed solution**
Add `send_chat_message({body, idempotencyKey})` and `POST
/api/agents/me/chat/messages`. The server selects the company, agent,
and responsible person from the active run. It saves an attributed
comment and audit record in one transaction. The chat shows a
source-task link and per-person unread state.
**Alternatives considered**
A destination parameter on a generic comment tool would expose
unnecessary routing authority. A new delivery worker would add another
durable delivery path. This change uses a dedicated tool and the
existing comment and live-event paths.
**Roadmap alignment**
This extends the experimental Agent Chat capability in `ROADMAP.md`. The
existing `enableAgentChat` setting controls it. Related public work:
Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat
and unread chat found no duplicate PR.
## What Changed
- Add a focused messaging service shared by the native tool and HTTP
route. It checks current run ownership, execution mode, recipient
membership, task access, and task privacy.
- Reuse the canonical company/agent/person conversation. Save the
comment and audit record together. Keep task completion and chat
execution separate from delivery.
- Add source-task provenance and source-scoped retry keys. Identical
retries return the saved receipt. Changed content returns a conflict.
- Enforce the conversation's fixed agent at the shared comment-write
boundary. Keep normal reply cancellation and session checks.
- Add chat unread state, accessible agent dots, aggregate Chat
indicators, and visible-message read cutoffs. Refresh through existing
live events.
- Add the request-key constraint, native tool contracts, documentation,
capability maps, stories, and regression coverage.
## Verification
- Local delivery/security proof: all 38 proactive-message cases pass.
Eight credential-scope regressions and four quota regressions fail
against the prior source. The repaired delivery and canonical
cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server
typecheck, and capability-map checks pass. The shared per-run cap covers
native and HTTP sends, mixed ordinary mutations, concurrent last-slot
contention, and retry/recovery without double charging.
- Credential tests preserve the authenticated person and authorizer;
reject bridge/test credentials and persisted skill-test runs after
task-mode changes; and recheck API-key revocation, scope, and
responsible person after destination locking. The earlier
delivery/native-authority suites passed all 74 cases before the four
quota additions.
- Four PostgreSQL contention cases cover native and HTTP sends plus
retries during a native chat mutation. All four deadlock against the
earlier source and pass with destination-before-source/run locking.
Recovery and human-reply delivery enqueue only the human comment; a
proactive digest never wakes the chat agent.
- Passed 108 companion conversation, native-authority, and OpenAPI
tests; 112 runner catalog/workflow-report tests; 47 completion-delivery
tests; and 25 project-handoff tests. Focused runner tests cover native
tool availability without generic API tools.
- All 327 affected UI tests, UI typecheck, token gates, and catalog
checks pass. The read hook retries at most three times, respects server
retry delay, coalesces only seen cutoffs, recovers on reconnect, and
cancels on scope changes.
- Real browser journeys with a deterministic process agent verify
routine-to-chat delivery, first-chat creation, unread state across
reload, the source link, secondary chat navigation, and a follow-up
reply referring to the digest. This does not qualify live GitHub
retrieval or model output quality.
- The Storybook showcase uses production components with deterministic
network fixtures. Rendered interactions pass for the landing, ordinary
read success, transient read recovery, and persistent failure after
bounded retries. Both Chat navigation variants update from two unread
chats to one while only Kodi's dot clears. Mobile and light variants
were reviewed. The existing Coordinator Recents label contrast finding
is unchanged; no Linux visual baseline archive exists.
- Current master's independent sandbox migration retains 0418.
Regenerated the feature migration as 0419 from the combined schema and
retained exactly the newest five snapshots; numbering/safety, replay,
and schema-drift checks pass.
- Workspace typecheck and build passed before PR preparation; canonical
server/UI typechecks passed again on the combined tree. Migration
`0419_slippery_impossible_man.sql` passes safety and schema drift checks
on disposable PostgreSQL, including interrupted index repair, concurrent
migrators, one-time owner read-state backfill, preserved receipts, and
two SQL replays. A default five-second local snapshot comparison timed
out; a diagnostic 30-second run passed without drift.
- Earlier failures are retained in the record: full local `pnpm
test:run` and broad shards encountered embedded-PostgreSQL startup,
timing, and provider-recovery failures. Thirteen selected
external-channel cases pass in isolation on the original and changed
sources; an unordered comment-row assertion now explicitly orders both
reads. The earlier route registry, native tool count, runner catalog,
and accessible Chat badge-name failures were fixed and passed full
remote CI on `920b3bb7db`.
- Repeated managed-skill server8 failures came from CI's newer synthetic
merge: its Cloud guard correctly rejected the old local-disk fixture.
Master independently repaired the fixture in #18385; this branch keeps
that repair unchanged. All 57 managed-skill and Cloud prerequisite cases
pass on the combined tree, and that fixture has no remaining PR diff.
- Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`:
Greptile 5/5 with no findings, Superagent security assessment passed,
advisory Paperclip Review 5/5 with no findings, and zero open review
threads. All 55 current-head check runs pass, with five expected skips
and the advisory review neutral at 5/5; Snyk also passes. No checks are
pending or failing. Protected squash auto-merge is enabled and is
waiting for required code-owner approval. A normal exact-head merge was
refused by the base-branch policy; no admin bypass was used. The
previous head's Superagent quota finding is fixed and covered above.
## Risks
- Chat remains company-visible. Delivery rejects a source task with
private-task, project, or ancestor restrictions.
- Read state uses a comment timestamp cutoff. The server retains
database precision and never moves a read receipt backward.
- The new unique constraint changes comment query planning. An existing
regression now uses an explicit row order. The migration builds the
unique index concurrently, then takes a brief table lock to attach it.
It uses the existing top-level migration and interrupted-index recovery
path.
- The upgrade backfills missing chat-owner read-state rows at the latest
existing agent comment. Existing read receipts remain intact. New chats
can still be unread before the human participates.
- Quota observation commits atomically with delivery. A rejected send
rolls back its attempted denial audit row and any first-chat creation;
the canonical warning remains in server logs, and previously spent quota
remains enforced. Identical/conflicting retries and receipt recovery
spend no extra slot.
- This version delivers in-app only. It adds no external-channel, push,
or email delivery.
- Local full-suite acceptance remains limited by the failures recorded
above. The feature stays behind `enableAgentChat`.
## Model Used
OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool
use and code execution. The earlier implementation also used Codex, but
its exact model ID was not recorded in the visible session. The
context-window size was not provided.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run focused tests locally and they pass; the broad local
failures are recorded above
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
DottaandPaperclip
|
7114582c63 |
fix(ci): track repository inputs across test projects (#18398)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - PR CI selects affected tests before allocating parallel runners. > - Recent audits retained about 96% of test files despite narrow changes. > - Unchanged consumer packages kept full coverage, and source scans confused data fields and message text with runtime loaders. > - File reads need explicit edges that imports alone cannot provide. > - This change follows consumer graphs and known file inputs while retaining unknown effects. > - The AWS CODEOWNERS gate stays in front of runner allocation. ## Linked Issues or Issue Description Refs #18376 and #18362. Narrow source changes still selected most tests because consumer packages kept full coverage and known file reads were treated as arbitrary input. Expected behavior is to retain affected import/file consumers and unknown inputs, then partition only the selected files. The shared storage failure found during the audit was fixed upstream in #18357. The candidate merge ran all 18 managed-skill storage tests successfully in [Linux qualification](https://github.com/paperclipai/paperclip/actions/runs/41140209697/job/124352365547). This branch includes that upstream commit; no additional storage implementation or permanent diagnostic job is included. ## What Changed - Assess every Vitest project in the coarse plan, including unchanged consumer workspaces. - Add dependency edges for statically resolved module-relative filesystem reads and directory listings. Keep unknown paths, writes, escaped APIs, mutable path objects and untracked inputs conservative. - Distinguish data keys, erased types and message strings from runtime globals, loaders and mock bypasses. Retain destructured globals, aliased object reads and mocked filesystem/path resolution. - Normalize `fs` and `node:fs` before recording coverage so a known read cannot hide an unknown one. - Review the Drizzle SQL/schema entry points by hashes of their complete import closures. CommonJS/dynamic loaders reject the fingerprint; drivers and migration readers remain opaque, and dependency updates revoke the review. - Record file input edges and per-test retention reasons in selection audit artifacts. - Add cross-workspace, mixed-import, symlink retargeting, setup-input and dependency-review regression cases. Object-valued URL constants are explicitly opaque with and without mutation. Update the developer guide. - Preserve the AWS CODEOWNERS authorization gate, runner-group restrictions and existing lane ownership. ## Verification - Final head `eedad362e84d6b68f0f048c92db6032e45622c46` passed [complete Actions CI](https://github.com/paperclipai/paperclip/actions/runs/41144233520) on attempt 1. Overall status: 57 successful checks, three intended skips, and one neutral Paperclip review. Both Greptile and Paperclip rated this head 5/5; zero unresolved review threads. - All 17 local selector tests passed: `node --test scripts/ci/test-selection*.test.mjs`. Final [Linux qualification](https://github.com/paperclipai/paperclip/actions/runs/41144233203/job/124365428832) passed all 18 selector/impact tests without skips, plus real workspace selection and cache/routing checks. - Same-tree local comparison against the previous selector at `d331d8481a`: Inbox changes selected 466 rather than 700 files (33% fewer); auth middleware changes selected 2,141 rather than 2,788 files (23% fewer). The sampled plans retained the same 2 and 24 jobs under existing weighted budgets. Final Linux qualification selected 466 files for Inbox and 1,949 for auth/config-test changes; platform-dependent graphs differ, so these are not a Linux before/after comparison. Chat/native Runner lanes retain their existing ownership and are excluded from these counts. - Local `pnpm -r typecheck`, `pnpm build` and `git diff --check` passed. The complete local `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL startup failures; initdb reported exhausted macOS semaphore capacity. Full Linux CI passed on the final combined tree. - The storage regression was corrected upstream in #18357. [Focused Linux verification](https://github.com/paperclipai/paperclip/actions/runs/41140209697/job/124352365547) ran all 18 managed-skill tests successfully, with no skips; the final full CI run also passed its server lane. ## Risks - Unknown effects must retain their consumers. Regression cases cover mixed builtin spellings, dynamic paths, escaped/mocked APIs, global aliases, reviewed dependency changes and cross-package file reads. - Graph analysis now visits unchanged consumers. Its added work must be included in rollout latency measurements; fewer test files do not by themselves prove shorter end-to-end PR time. - The weighted shard budget deliberately remains unchanged. These samples reduce file execution without reducing allocated jobs. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code execution and GitHub tools. The exact serving variant and context window are not exposed in this session. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — targeted selector suite; full local suite limitation documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
github-actions[bot]andlockfile-bot
|
4eaa435bf4 |
chore(lockfile): refresh pnpm-lock.yaml (#18414)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com>canary/v2026.1031.0-canary.2 |
||
|
Devin Foley
|
cdf843371b |
fix(runner): build on Windows and add a Windows CI lane (#18250)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Paperclip Runner (`packages/paperclip-runner`) runs agent sessions
through a native Rust daemon, `paperclip-runnerd`
> - The daemon builds only for macOS and Linux. On Windows x64
(`x86_64-pc-windows-msvc`), the build stops with one compile error
> - No CI lane builds the daemon on Windows, so new Unix-only code can
break the Windows build again without notice
> - This pull request fixes the compile error and adds a Windows CI lane
for the Rust workspace. It is the first of a series of small pull
requests that port the Runner to Windows
> - The benefit is a Windows build that stays green. Each later pull
request in the series can then prove its Windows behavior in CI
## Linked Issues or Issue Description
Related: https://github.com/paperclipai/paperclip/issues/18142 and
https://github.com/paperclipai/paperclip/pull/18144 port the server and
the development scripts to native Windows. That work does not touch the
Rust Runner.
**What happened?**
On Windows x64 with the pinned Rust 1.97.1 MSVC toolchain, `cargo build
--manifest-path packages/paperclip-runner/runner/Cargo.toml --locked
--workspace` fails with `error[E0599]: no method named inherited_command
found` in `process_supervisor.rs`. `VerifiedProcessLaunch::spawn_stdio`
calls `inherited_command`, but that method exists only on Linux and
macOS.
**Expected behavior**
The Runner workspace builds on Windows x64. Features that do not have a
Windows implementation yet fail with a clear error at run time. They do
not stop the build.
**Steps to reproduce**
1. Use a Windows x64 host with Visual Studio 2022 Build Tools and
rustup.
2. Check out `master`.
3. Run `cargo build --manifest-path
packages/paperclip-runner/runner/Cargo.toml --locked --workspace`.
**Paperclip version or commit**
`master` at
|
||
|
Nicky LeachandPaperclip
|
493e229d93 |
perf(runner): cache verified native executable digests (#18390)
## Thinking Path > - Paperclip manages AI agents and their task execution. > - Runner checks qualified native programs before it starts a provider. > - Installation checks and command opening can hash the same program again. > - A new sidecar process repeats this work on a later warm task. > - This pull request stores the computed digest in a private filesystem cache. > - Every cache hit still checks the independently selected trusted digest and current file metadata. > - An unchanged program can start without another full native file read. ## Linked Issues or Issue Description **What existing behavior does this improve?** Repeated **qualified ACPX** native executable admission in Runner. Claude, ACPX-Codex, and Grok use this verifier in both Core-owned and plugin-backed launches. The ordinary native Codex app-server launcher is a different path and is not changed. A retained provider that opens no new native command has no saved hash work here. The measured Claude task path does not require a comment backlog. **Current behavior** Installation verification and final command opening each read and hash the native executable. The qualified Linux x64 Claude file is 241,667,256 bytes. A new verification and command lease reads 483,334,512 native bytes. Another Node process repeats both reads. **Proposed behavior** Store the actual computed SHA-256 digest by device, inode, size, nanosecond modification time, and nanosecond change time. Reuse it across Runner and sidecar processes. Compare each hit with the current build-owned digest. Hash the file on a miss or changed identity. This replaces the earlier Claude-only deferral design in this same PR. **Reason and benefit** The first verification and command lease reads the native file once. Later processes read none of its native bytes while its identity stays the same. Related: #16372 established qualified executable admission. This change is independent of other preparation PRs. It does not change provider pins or profile versions. ## What Changed - Add a shared filesystem digest cache under the absolute OS temporary root at `paperclip-runner-<uid>/executable-digests-v1`. - Use private directories and files, versioned hash-only filenames, bounded JSON entries, and atomic temporary-file replacement. Cache files contain a version and the computed digest. They contain no credentials or source paths. - Fall back to full hashing when the cache is missing, unreadable, malformed, or unusable. Keep current expected-digest selection outside the cache. - Keep cache data outside task homes. Neither `HOME` nor task cwd selects the path. The OS temporary-root selection honors absolute platform `TMPDIR`, `TMP`, or `TEMP` overrides. A relative root disables caching. Hosts must keep that root outside task workspaces. Temp cleanup causes a later rehash. - Keep the no-follow executable open, regular-file and size checks, execute permission, descriptor ownership, before/after identity checks, and preparation-to-launch identity comparison. Cache publication occurs before the final identity checks. - Use this path for standalone installation checks and final command opening. Remove the earlier pending-digest type and Claude-only launch helper. Runtime host and profile selection keep their original structure. - Test changed content, all fingerprint fields, symlinks, mode changes, inode replacement, mutation during lookup/publication, concurrent cache writers, wrong trusted pins, and cache failures. - Include a deliberate trust-boundary test: a process running as the same OS user can plant a matching cached digest for changed bytes. It opens a test descriptor and executes nothing. In plain language: Runner remembers the hash for the same file. It still checks which file it opened and which hash the build requires. The cache trusts the OS user who can write it. ## Verification From `packages/paperclip-runner`: - `../../node_modules/.bin/vitest run src/drivers/acpx/executable-digest-cache.test.ts src/drivers/acpx/installation-integrity.test.ts src/drivers/acpx/runtime-host.test.ts` — 177 passed, 6 platform-specific skips. - `../../node_modules/.bin/tsc -p tsconfig.json --noEmit` — passed. - `git diff --check` and the added-line secret, PII, private-address, and code-comment scan — passed. The offline comparison uses baseline `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9` and treatment `c4356df12032448de90516ae329d8f939af10984`. Each sample starts a different Node process. Each normal sample runs the actual qualified installation verifier and opens a command lease. Seven samples per case follow one warmup. The cache is removed before each cold case. The next process reuses its entry. Baseline order alternates around the treatment pair. Command digest and package paths match in every pair. | Verifier plus command opening | Baseline | Cold cache | Later process hit | | --- | ---: | ---: | ---: | | Full native reads | 2 | 1 | 0 | | Native bytes read | 483,334,512 | 241,667,256 | 0 | | Local median, ms | 1041.26 | 602.70 | 130.17 | | Local range, ms | 994.20–1133.11 | 580.27–999.83 | 121.96–179.45 | | Median whole child process, ms | 1379.96 | 988.75 | 523.01 | Raw verifier-plus-open samples in ms: - Baseline: 1133.11, 1074.40, 1061.99, 1041.26, 994.20, 1036.12, 1037.70. - Cold: 647.13, 646.33, 999.83, 584.83, 580.27, 583.37, 602.70. - Later process hit: 179.45, 138.37, 134.39, 130.17, 121.96, 122.33, 126.68. Separate direct-opener fault probes use a disposable copy of that same native file: | Change | Native bytes read | Result | Local open, ms | | --- | ---: | --- | ---: | | First copied file | 241,667,256 | Accepted | 526.14 | | Next process, different HOME and task cwd | 0 | Accepted | 24.91 | | Changed timestamps | 241,667,256 | Accepted after hashing | 531.40 | | New inode, identical bytes | 241,667,256 | Accepted after hashing | 536.70 | | Corrupt byte, same size and exactly restored mtime | 241,667,256 | Rejected | Whole process: 841.36 | | Different independent trusted pin | 0 | Rejected | Whole process: 374.76 | The corruption probe keeps `mtimeNs` exactly equal before and after, at `1000000000000000000`. Its changed `ctimeNs` invalidates the entry. Cache expiry is not required. The changed-pin probe rejects the old actual digest without a native read. The cached image is `sha256:99ce80690a0c13328f41fd84156cc73e1ac562eaed3120fd41d04d7acdfb325d`. The Node binary is `/opt/paperclip/runner/libexec/node`, version `v24.20.0`. The artifact is Claude SDK `0.3.286`, with expected digest `sha256:fe503f65c6289d59c23e5b21ae44f03583f997dd33a2cbfc75ab4f96fb8fc73f`. The container used a read-only root, a disposable writable tmpfs, and disabled network. No provider process, model request, credential, or deployment was used. **These elapsed values are diagnostics from Linux x64 CPU emulation on an ARM64 Docker host. They are not production latency or TTFT estimates.** The exact byte counts, cross-process reuse, output equality, and invalidation results are the portable evidence. The saved phase 0 exports have no directly comparable native executable hashing span. Their broad `runner.transport.attach` spans include other work. A production comparison within 20% is therefore unavailable for this seam. The earlier local 919 ms control was also an emulation diagnostic; it is not a production baseline. The initial Docker attempt was denied access to the local socket by the sandbox. The same offline command then ran with local socket permission. An initial fault probe restored mtime through a `Date`, which did not prove exact nanosecond equality. Its raw report is retained as historical; the final probe sets and verifies an exact mtime before and after corruption. The prior Claude-only implementation and its checks and measurements remain historical evidence. They do not qualify this replacement. The superseded home-directory cache head had one browser recovery test timeout in its GitHub workflow. It expected a saved title after session recovery. The failure is retained. No browser code was changed; the current head reruns CI after moving the cache outside task homes. No full local CI suite was run, as requested. GitHub CI supplies full regression coverage. Current head `c4356df12032448de90516ae329d8f939af10984` passed the complete [GitHub workflow](https://github.com/paperclipai/paperclip/actions/runs/41145223331), including all browser shards. Its latest checks have 56 successes and 2 intended skips, including both security scans started when the PR became ready. Standard Greptile reviewed this exact head at 5/5 with no open findings. Apex then reviewed the same head at 5/5 under the documented OS-user trust policy. It reported no new findings. There are no unresolved review threads. The PR is ready for review and remains unmerged. ## Risks - This deliberately weakens protection against the same OS user. That user can write the cache and can plant a chosen digest for a file fingerprint. A cache hit trusts that record; metadata identity is not a fresh content hash. The explicit fault test documents this accepted policy. - A read-only administrator-owned Runner can use this user-writable cache. The cache has no signature, secret, or daemon. It does not require a Launcher or ROM change. - Other users cannot use normal cache permissions to write these entries. Cache errors fall back to hashing. Removing the cache directory restores full hashing. Old fingerprints leave small unused entries until the cache is removed or OS temp cleanup evicts it. Temp eviction costs one later full hash. - A changed file normally rehashes. A digest that differs from the independent current pin still fails admission. Existing command, package, credential, process-lifetime, and final dispatch checks remain. - No schema, public protocol, package dependency, profile version, or provider pin changes. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, code editing, test execution, and tool use. This session does not expose a more precise serving model ID or context-window capacity. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the details available to this session - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] The targeted local tests pass; the full local suite was not run as requested - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green for the replacement head - [x] Standard Greptile is 5/5 with no open findings for the replacement head - [x] Apex is 5/5 on the current head with no unresolved findings - [x] I will address review findings before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1031.0-canary.1 |
||
|
Devin FoleyandPaperclip
|
87ae20ae65 |
fix(server): reconcile failed sandbox cleanup with signed absence (#18407)
Reconcile failed ephemeral Cloud leases through a signed cleanup-only absence observation when local historical inspection is unavailable. Preserve failure outcomes, exact resource bindings, retention and row-version fences; grant no run-continuation authority. All 54 current-head checks/statuses passed. Greptile is 5/5, Medium, with no findings or unresolved threads. The 38 focused tests pass; both new reconciliation cases fail without the fallback. Isolated baseline and branch idle suites each pass all 225 cases. The original broader local teardown timeout is retained in the PR, and the full local run remains in progress. Cloud protocol support is already serving in staging. Exact candidate composition and hosted provider sleep/wake qualification follow this merge. Production has not been changed. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
105a67b52c |
Reuse signed master images for npm canaries (#18404)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Release workflows publish npm packages and multi-platform images. > - A master push can finish its image before the npm canary dispatch starts. > - The dispatch currently builds that same source again on two runners. > - This PR verifies and reuses the signed master image by digest. > - Missing images retain the native build path. Channel promotion still follows npm. ## Linked Issues or Issue Description **What existing behavior does this improve?** The Docker workflow dispatched after npm canary publication. **Current behavior** Every canary dispatch builds both architectures, even when master has already published the same source. This consumes runner capacity and delays the image channel update. **Proposed behavior** Resolve the full-SHA master image. Verify its platform index, digest and signed source identity. Retag that immutable digest. Build normally when the image does not exist. **Reason and benefit** Remove duplicate image builds while keeping missing-image recovery and the current npm channel check. A recent duplicate used about 25 runner-minutes. The saving depends on whether the master image exists. ## What Changed - Add a canary-only reuse job before the native matrix. - Require signed provenance from the canonical Docker workflow on master at the exact source and signer commit. Read the proof from GHCR. - Reuse the validated digest or build both platforms when the full-SHA image is absent. Stop on registry and verification errors. - Keep the serialized npm dist-tag promotion after either path. Keep other build lanes independent. - Add failure tests, register them in CI and update release documentation. ## Verification - Passed 603 tests: `node --test '.github/scripts/tests/*.test.mjs' scripts/standard-image-contract.test.mjs scripts/reuse-canary-image.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/cloud-source-verification.test.mjs`. - Passed `actionlint -shellcheck='' .github/workflows/docker.yml .github/workflows/pr-trusted.yml` and `git diff --check`. - Full actionlint reports the same six existing shellcheck findings as the unchanged master workflow. No new findings. - A published master image passed the exact provenance command. This was a read-only check. No images were retagged. - Gitleaks and manual diff inspection found no secrets or private references. - Attempted `pnpm -r typecheck`, `pnpm test:run` and `pnpm build`. They cannot start because this isolated checkout has no installed dependencies (`tsc` and `tsx` are absent). Repository CI passed on `3c46f0152d4db94a7f393bc79f466aa6c2c745e2` after the documented browser-test rerun. All checks are complete. The original failure and prior baseline occurrence remain recorded in the PR comments; its root cause remains unknown. ## Risks - The image can appear just before its attestation. Verification then fails closed. Rerun after the master producer completes. - Reuse preserves the first build's resolved dependencies and tool versions. It intentionally does not rebuild the same source to refresh floating inputs. - A skipped or unfinished master image still needs a full build. No speedup is claimed for that path. - End-to-end reuse and runner savings require observation after merge. No live publication was performed. ## Model Used OpenAI `gpt-6.1-sol`, through the Codex harness in T3 Code, with xhigh reasoning effort and shell/code tools. The context window size is not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
9e904ee519 | perf(server): overlap independent heartbeat preparation reads (#18372) | ||
|
DottaandPaperclip
|
1353a232f2 |
test(e2e): add a portable active follow-up journey (#18345)
## Thinking Path > - Paperclip lets people direct agents while their work is active. > - Product E2E checks must prove that saved user input reaches the intended work. > - The existing active chat story depends on a file in a local agent workspace. > - That setup cannot qualify an attached deployment with remote agent execution. > - This change adds one shared active follow-up journey through the normal task UI and API. > - Independent native operation, saved input, final reply, and cleanup checks make the result inspectable across targets. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for a user follow-up sent while a native agent is working. **Current behavior** The classic chat story holds work open with a local file. The shared deployment-targeted harness has no active follow-up journey. A running heartbeat alone cannot prove that work was active when input arrived. **Proposed behavior** Run one bounded command through the real native provider. Observe its active execution before and after one follow-up is saved. Require one final reply with the late requirement, correct task and agent identity, an empty input queue, and visible readback after reload. Use the same story on managed and attached targets. ## What Changed - Add a shared `followup-while-running` journey with normal task creation and reply submission. - Require a bound native turn and pending shell execution at the input boundary. Retain a closed receipt with source identity and a row hash. - Accept one steered work run or one normal successor (1–2 native runs). Preserve every other journey's declared run contract. - Reuse the saved native final reply observer, run binding checks, usage collection, target lifecycle and private evidence path. - Accept canonical builtin/process execution events independent of provider tool display names; calibrate against real OpenCode, Codex and ACP mappers. Reject Paperclip semantic tool names across native wrappers/prefixes and unnamed or ambiguous builtin calls. - Recognize only the observed legacy PRP v1 builtin/bash/unknown/readOnly=false command shape and fail promptly when owned work finishes before a live boundary can be observed. - Ready the task page/composer before the observed command boundary, use a bounded 90-second command, and preserve active-after-persistence proof. - Add adversarial oracle and retained-result calibration. Register the journey, include its sources in the definition hash, and document its coverage. - Keep six real browser document-readiness checks and verify shared-deadline behavior with a deterministic clock/state test that rejects timeout-reset mutations. ## Verification Exact current head `e72e20ad111bd94728d38606015d1fdac3c92929` passed [full CI run 41142552195](https://github.com/paperclipai/paperclip/actions/runs/41142552195): all 49 workflow jobs reached accepted terminal results, including typecheck, build, every test lane and aggregate verify. All 59 reported checks are terminal: 55 success, three expected skips and one neutral 5/5 Paperclip review. Snyk passed. [Fresh Greptile review](https://github.com/paperclipai/paperclip/pull/18345#issuecomment-6528913235) explicitly names this exact head and gives 5/5. No review threads remain open. The PR is mergeable. These source gates do not establish a live QA pass at this head. Fresh protected QA at this exact source and the separately recorded deployed application/runtime remains required before primary merge and daily admission. Historical live QA staging **PASS** on qualified commit `af9e091b15b5e24b6b2cc1b424523430cc2ab024`: 6/6 checks, all five phases including cleanup, two native work runs, and the configured model/environment check passed. The [sanitized qualification report](https://pages.paperclip.ing/evals-qa-smoke-41122620146-1/) records that exact measurement. Final integration head `e72e20ad111bd94728d38606015d1fdac3c92929` merges exact actual squash `18b1637b6371a6a6adade857e2a73654ee626fef` from [#18254](https://github.com/paperclipai/paperclip/pull/18254) into previous head `e54d64743964b1a19d165537bebbe62c50389a23`. The merge had no conflicts. All 18 files in this PR's change set, the complete 45-file shared journey directory, and six shared native/evidence helpers retain their exact previous-head Git blobs: 59 scoped files in total. The production clarification helper remains blob `800a747284db19b8484650b6759e8bd1f356839d`. Follow-up 1–2 runs, question 1–3 runs, older exact counts, owned native event checks, active-before/after persistence proof, cleanup/accounting and feature mappings remain unchanged. Source integration head `e55d74491f5bcd99c1455e9ace02bd7ad7319fae` integrates exact actual squash `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` from [#18385](https://github.com/paperclipai/paperclip/pull/18385). The merge had no conflicts. All 17 files in this PR's change set and six shared native/evidence helpers retain their exact Git blobs from previous head `162e341cda5b627639b486b4019a1675bb018ea2`. This preserves the follow-up and question readiness checks, feature mappings, six closed follow-up check IDs, immutable before/after active-operation proof, follow-up 1–2 runs, question 1–3 runs and older exact counts. Previously reviewed head `e54d64743964b1a19d165537bebbe62c50389a23` adds a separate test-only reliability repair on top of that source integration. It replaces one scheduler-sensitive browser case with a deterministic Vitest clock and locator-state model that calls the actual production helper. The six other real Chromium checks remain byte-for-byte unchanged and pass **6/6**. Focused unit suites pass **51/51**, including the deterministic deadline case. This is six browser checks plus focused unit coverage, not seven browser checks. The model makes the panel ready at 100 ms, the supplied answer ready at 600 ms, and full valid content ready at 1,300 ms. It requires rejection at the shared 1,000 ms deadline and records immutable rejection state before simulating a 10-second scheduler stall. Both restarted-timeout mutations fail: resetting all waits wrongly resolves at 1,300 ms; resetting matcher timeouts while keeping the final real deadline guard rejects only at 1,300 ms with valid content present. The genuine helper rejects at 1,000 ms with the answer still incomplete. The production helper is unchanged, Git blob `800a747284db19b8484650b6759e8bd1f356839d`. That two-file test-only patch changed no journey, application, run, accounting, model or runtime behavior. Its six-browser/51-unit results are prior-head evidence; the complete patch is unchanged in the new integration. The final source also retains the actual merged question-primary squash `d9354bbfc35ef74333e90e596ce21dc8e59149b9` from [#18346](https://github.com/paperclipai/paperclip/pull/18346). Later commits `409117451da260a03fc28a3107c7808ecbe68f8f` and `ea3e7b1d1bed562351fcf4722c7175001b987b59` bound question document readiness to one campaign deadline and test that waits cannot restart it. Commit `162e341cda5b627639b486b4019a1675bb018ea2` adds question/follow-up report feature mappings and coverage. These files are unchanged by the final merge. The imported upstream source has real application changes: [#18334](https://github.com/paperclipai/paperclip/pull/18334) makes managed skill content durable and extends native skill-test contracts; [#18350](https://github.com/paperclipai/paperclip/pull/18350) requires managed Cloud shared storage and bootstrap prerequisites. #18385 updates TestOps job metadata and repairs their combined maintenance fixture. The new merge additionally imports [#18383](https://github.com/paperclipai/paperclip/pull/18383) at `81ecaf0654fe79a2a2dbda6588f033e39ee610e4`, which bounds failed lifecycle verification; [#18357](https://github.com/paperclipai/paperclip/pull/18357) at `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`, which persists sandbox-creation ownership and updates cleanup/adapter/plugin contracts; and #18254 at `18b1637b6371a6a6adade857e2a73654ee626fef`, which accepts a valid late Antigravity authorization URL while native authentication drains. This PR's diff against exact upstream `18b1637b6371a6a6adade857e2a73654ee626fef` contains only journey tests, report coverage and documentation. Harness source and deployed application/runtime identity are separate inputs. This source integration does not deploy the imported application changes or claim the QA target runs this application revision. This final SHA has not been measured by the historical run above: **fresh QA at this exact final head is required before primary merge**, followed by normal TestOps verification/admission. Fresh exact-head CI and review have passed as recorded above. The live measurement must record the actual separately deployed QA application/runtime identity. Seven whole-file blobs remain identical to the qualified SHA. All paths are relative to `tests/runner-e2e/`: | Journey/helper | Unchanged Git blob | | --- | --- | | `product-journeys/active-followup-evidence.ts` | `83a5d9fab52abb9849a40e83eca6b6a9c29bf587` | | `product-journeys/active-followup.test.ts` | `728160c610c18292d2cf06d4a87d25a8719f1734` | | `native-task-reply.ts` | `59d11f1a178dad9db3da8fa2342ff93643cb9922` | | `native-event-envelope.ts` | `305b0b456c59c511ee328e529ad574c0583075a0` | | `run-observations.ts` | `08c8ad184a821a7cd536926b3d7e3d1156a1e3da` | | `user-actions.ts` | `e9611348dc41e17899367dab2e3b36d3d1be9480` | | `stop-redirect-flow.ts` | `4580bd605a425bc72511e02d988492acbec8d60e` | The follow-up preservation comparison against historical `af9` has three explicit exceptions; no whole-file identity is claimed for them. The separately documented question readiness and report additions above also require final-source verification: 1. `product-journeys/agent-flows.ts` adds `question-answer-resume` to the `AgentJourney` type exclusion, alongside follow-up. Its remaining contents and runtime helper bodies are byte-for-byte unchanged. Qualified blob `a0afaa968c1d854476159fef21de7b6dd50a4a98`; final blob `0cc797c0ae594e7769338e63a80c4f67e29e4a54`. 2. The common run-count helper is centralized in `success-contract.ts` and re-exported by the catalog. It accepts follow-up **1–2**, clarification **1–3**, and each older paid journey's exact count. The follow-up catalog entry, maximum of two runs, six required checks, native dispatch, collection and retained-run contract remain byte-for-byte unchanged. The helper's text differs; its preserved count semantics are verified explicitly. 3. `product-journeys/active-followup-flow.ts` changes exactly one composer readiness assertion from Playwright's implicit five-second default to an explicit 30-second timeout, matching `submitTaskReply` for attached targets. Prompts, actions, native evidence, before/after persistence proof and grading otherwise compare byte-for-byte unchanged. Qualified blob `64392329198a1a4d9828e32834d4507e175216e6`; final blob `2746b5ba89e2af26b4097ba7897913186786ac5d`. This correction is why a fresh final-head QA run is required. - **310 focused tests passed at current final head `e72e20ad111bd94728d38606015d1fdac3c92929`**: 286 journey/evidence/catalog/report/shared-deadline tests in 14 files and 24 native authentication/real-child stream-order tests in two files. Both invocations used Node24 and one Vitest worker. The earlier `e55` integration also passed 365 focused tests, including 40 Cloud configuration and 40 TestOps metadata cases; those results remain prior-source evidence. - **157 explicit count assertions passed again at the current final head**, including both interactive ranges, older exact counts, invalid/fractional/non-number values and catalog/contract export identity. The maximum bounds and six closed follow-up check IDs also matched. - Maps and diff checks passed. The official QA runtime-import preflight passed on Node24 at the current final head without network access. It uses a dummy adapter and no credentials. It does not prove target connectivity, sign-in, model readiness or deployed application/runtime identity. Those require the normal protected QA run. No model/provider work or optional Runner release was dispatched. - Current local runtime dependencies use ignored isolated workspace-source views. No shared dependencies or compilers were changed. A previous local typecheck attempt hit stale reused Runner declarations. No full repository typecheck, test run or build was repeated locally; the successful exact-head full CI run above now supplies these gates. - CI on previous head `162e341cda5b627639b486b4019a1675bb018ea2` failed the Cloud/local maintenance fixture in [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133716802/job/124331643100), plus dependent verify. The actual tested merge combined that head with master `198218bf554877fe3b7a83742eefb3b8f12392c6`. A Cloud child selected local-disk storage and hit the new shared-storage policy. This was a deterministic upstream fixture mismatch; no unchanged failed-job retry was used. - The actual #18385 repair configures a valid Cloud child, verifies local bytes independently, checks the expected provider mismatch and retains exact stored-config readback. Its own final head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57` passed 59 checks, including [server shard 9](https://github.com/paperclipai/paperclip/actions/runs/41133876636/job/124331450635). That prior result supports the repair; it does not replace this final head's fresh CI. Separate repair candidate #18392 and its source/evidence are retained as superseded history. - A local attempt to run the database fixture could not start PostgreSQL because the host exhausted macOS semaphores. A first setup attempt also failed package resolution before test execution. Skipped database cases and these setup failures are not passes. No kernel settings, shared processes or databases were changed. - The complete previous-head CI run [41138618869](https://github.com/paperclipai/paperclip/actions/runs/41138618869) finished with one underlying failure: [Runner Vitest shard 1](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445) expected native Antigravity login success but observed failure at `antigravity-auth.test.ts:101`. The lane had one failed test, 1,994 passing tests and eight skipped tests. Aggregate verify failed only because of this lane. All other 57 checks were accepted, including build, typecheck, all server/browser suites, Runner shard 2, Snyk and exact-head 5/5 reviews. No unchanged retry was requested. - The independently reviewed #18254 repair removes an invalid cross-pipe ordering assumption: a valid URL on stderr can arrive after the authenticate response on stdout during drain. Its real-child regression forces both orders. The current final source imports that actual squash; the 24 focused authentication cases pass here. Fresh full CI and exact-head review have now passed for this source, as recorded above. The earlier failed result is not relabeled. - The original live compatibility failure is retained: an older PRP v1 pack labeled builtin `bash` as operation `unknown`, causing the observer to miss the command and never send the follow-up. Its exact shape was reproduced red before repair. Generic unknown operations, semantic tools, wrappers, read-only work and foreign provenance remain rejected. The historical qualified pass used the repaired oracle; no failed result was relabeled. - Earlier CI on `6fad965d69` hit an unchanged heartbeat test's three-second adapter-call wait; its original failure and one narrow rerun remain recorded. The qualified SHA subsequently passed CI. An earlier report-catalog subprocess failure is retained. The final focused Node24 run now includes that suite and passes after the documented report feature mappings. ## Risks The fixture asks the native agent to run one bounded `sleep 90` command. The observer must see actual active native execution; a fast or noncompliant turn fails rather than passing from elapsed time. A completed result requires one or two bounded native work runs, source attribution, configured model/environment, and collection. Unknown provider prices stay unknown. This change does not alter production execution or grant new access. ## Model Used OpenAI Codex, GPT-6 family, with reasoning and code execution. The host does not expose an exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with available version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs; existing follow-up execution fixes do not provide this deployment-targeted test - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket or instance-derived details - [x] I have run the new focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
91261d5a03 |
fix(server): preserve budget history for deleted resources (#18389)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Budget views show saved policies, incidents and spend for each company. > - A policy can remain after its agent or project has been deleted. > - The overview currently stops when it cannot resolve that old resource. > - A current agent's budget then becomes unreadable during normal setup. > - This pull request preserves the historical row in display summaries. > - Admission and policy changes still require a real company-owned resource. ## Linked Issues or Issue Description **What happened?** The budget overview throws `Agent not found` when a saved policy references a deleted agent. One old resource prevents the company from reading its current budgets. The same problem affects historical project policies and incident summaries. **Expected behavior** The overview should retain historical policy and incident IDs and spend. It should display `Deleted agent` or `Deleted project` for a missing resource. Read access must not grant permission to execute work or change a budget for that resource. **Steps to reproduce** 1. Create an agent and a budget policy for it. 2. Permanently delete the agent after its other dependencies have been cleared. 3. Read the company's budget overview. The existing service throws instead of returning its remaining current and historical rows. **Paperclip version or commit** The isolated repair starts from merged commit `d9354bbfc35ef74333e90e596ce21dc8e59149b9` and integrates the exact merged baseline test fix `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. It also integrates the actual merged Antigravity auth fix `18b1637b6371a6a6adade857e2a73654ee626fef` (#18254). The diff against the selected merged parent remains two budget files. **Related work** Earlier reports and implementations: #11058 and #4333. This extracts the product prerequisite from #18351 so it can qualify independently of that journey's other dependencies. The change contains only the budget service repair and four database regression cases. ## What Changed - Resolve deleted agent and project scopes only when building display summaries. Preserve their original IDs, policies, incidents and accounting records. - Reject a saved policy or incident that resolves to another company's resource. Propagate other read failures. - Keep missing-company resolution, work admission and policy changes strict. Validate company, policy and incident scope bindings before either incident action. A budget raise requires a live company-owned resource. Historical dismissal may proceed only for a genuinely deleted owned agent/project, and must reject live foreign or mismatched scopes. - Add four canonical PostgreSQL regression cases: deleted agent history, deleted project history, missing-scope admission and budget raises with safe historical dismissal, and foreign/mismatched references. The history cases also verify current ledger spend and unchanged stored rows. ## Verification - Real-service RED/GREEN against official PGlite with every canonical migration: the original overview throws for deleted history; the repair returns the historical IDs, incidents and spend while retaining current budgets. - Mutation regression RED/GREEN: missing-resource budget raises must reject before writes; owned deleted-scope dismissal must reject its linked pending approval, retain policy/agent/ledger data and incident history, and remove the incident from the active list. Live foreign and mismatched scopes reject both actions without mutations. - Current integrated head `b3434f5ad05fd3067379352700c36bfe826179c4`: all four canonical PostgreSQL regressions pass in [server 2/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194077), and all five existing budget-service tests pass in [server 7/12](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124361194080). Both use the CI merge against selected parent `18b1637`. - Clean-build declaration typing passes in the current CI typecheck gate. The earlier local source-alias no-emit check also passed, including the canonical actor argument. The local worktree borrows compiled dependencies from a sibling checkout; a later real-service import was blocked by its stale `sandboxCreationOperations` export, so clean-build CI is authoritative for the final integration. No production workaround was made. - `git diff --check` passes. Broad repository typecheck, tests and build run in current-head CI rather than repeated locally. Native local PostgreSQL could not start because this host exhausted its semaphore pool; no global IPC settings or unrelated processes were changed. - One unchanged Runner lane retry was authorized after a probable intermittent close-settlement fixture failure. The [original failure](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124360234111) occurred in `omitted-context` cleanup. A [fresh passing Linux lane](https://github.com/paperclipai/paperclip/actions/runs/41142552195/job/124360110386) passed the same case and all 1,998 tests with byte-identical entire Runner and adapter-utils trees, lockfile, root package, scripts and workflows, against the same merged base. The fixture has a five-second close allowance with a 2.5-second suspension reserve; a host stall is not proven. No budget path or changed close input was found. The [single targeted retry](https://github.com/paperclipai/paperclip/actions/runs/41142588583/job/124365812609) passed the same case in 1,034 ms and all 1,998 lane tests; dependent verification also passed. All 57 current PR checks are terminal and green. - The product helper requires all current CI and clean review before merge. Actual QA budget-overview proof will run on the combined final app before hiring qualification. No deployed success is claimed. The older `b4863dd` artifact was never deployed; selected parent `18b1637` adds sandbox/lifecycle changes and migration 0418, so whole-app identity is not claimed. The budget module and relevant budget contracts remain unchanged, and final hiring requires the combined final app with explicit source identities. ## Risks Budget summaries now return historical rows whose resources have been deleted. These rows use explicit display names and preserve their accounting data. They do not create a resource or relax work admission, authorization or budget grants. Historical incident dismissal changes only that incident and its linked approval; it does not resume work or raise a budget. Cross-company references still fail closed. There is no schema migration and no new background work. ## Model Used OpenAI Codex, GPT-6 family. The session does not expose a more specific backend model ID or fixed context window. Used reasoning, repository tools, TypeScript and database test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the PR description documents the display-only history and strict-mutation contract) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1031.0-canary.0 |
||
|
DottaandPaperclip
|
18b1637b63 |
fix(runner): accept late Antigravity login URL during drain (#18254)
## Thinking Path > - Paperclip manages work performed by AI agents. > - The native Runner owns Antigravity subscription sign-in. > - Sign-in receives completion on stdout and the browser URL on stderr. > - The two pipes can deliver the completion result before the valid URL. > - The owner rejected that URL after it entered the completed phase. > - This pull request accepts one validated URL while the same child drains. > - A real child regression now proves the order and the failure controls. ## Linked Issues or Issue Description Refs #18188, #18189, and #18190. Related qualification work: #18345 and #18356. The duplicate search found this existing PR, so this update preserves its history and adds the real child proof here. **What happened?** A valid sign-in can fail when the authenticate response arrives before the authorization URL on the other pipe. The completed phase then rejects the valid URL. The same existing success case failed in [current follow-up CI](https://github.com/paperclipai/paperclip/actions/runs/41138618869/job/124347859445). That log has no stream-order trace, so it does not prove its inner failure branch. **Expected behavior** Both valid delivery orders must succeed. Invalid or duplicate URLs, unexpected protocol requests, failed retirement, and failed credential verification must still fail. **Steps to reproduce** 1. Run a synthetic native child that completes authenticate on stdout. 2. Wait for stdin EOF before it emits the authorization URL on stderr. EOF proves that the parent accepted the result and began retirement. 3. The valid case fails on frozen master `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`. The invalid and duplicate controls pass there. 4. Run the same regression with this PR's existing production fix. All three cases pass. **Paperclip version or commit** Existing reviewed PR parent: `646f5cdf37898ad54e1fa1cdfe39f59844da3e7c`. Frozen actual master for the real child RED: `45a3a92e6dc52d20923b9aa34a22fee2ec5443f9`. ## What Changed - Accept one fully validated authorization URL in the authentication or completed-drain phase. Keep duplicate rejection and all existing URL checks. - Retain the ten existing in-memory controls for both valid orders and eight failure paths. - Add three real child cases for valid, invalid, and duplicate late URLs. Use stdin EOF to order the streams without sleeps or a mocked retirement handler. - Check exact protocol requests, child exit, close-once behavior, Google-call counts, and persisted refresh rotation. This update changes only the existing test file; it adds no production delta to the reviewed parent. ## Verification - Frozen actual master: the new valid real child case fails, while both negative controls pass. The original eight-control parent calibration also remains recorded in this PR's history. - Existing fixed source plus the new tests: 24 passed. This includes 11 existing authentication tests, ten stream controls, and three real child cases. - Late invalid and duplicate URLs fail with zero Google calls. The valid case makes two mocked identity calls and saves the rotated synthetic refresh token. All three retire the real child and close the command lease once. - Scoped owning TypeScript `noEmit` passes for the auth module and both test files. The temporary configuration is not committed. Diff and credential-pattern checks pass. - The tests use real child pipes and private synthetic files. Google responses and the command installation are mocked. No provider account, network request, model run, app, or QA deployment was used. - Full monorepo typecheck, tests, and build were not run locally. Fresh [normal CI](https://github.com/paperclipai/paperclip/actions/runs/41140757050) passes on `1744c9f3374f78b10c2f15c42f3cf5d3af8c3119`: 56 successful checks, two skipped, and one neutral. Typecheck, build, Rust, static, Runner, server and browser lanes pass. Greptile gives that exact head a fresh 5/5 with zero unresolved threads. The PR is mergeable and clean. The earlier parent's passing CI remains historical evidence only. ## Risks The change accepts a URL while the same authenticated child drains. It still validates the exact OAuth URL and rejects a second URL. Success still requires the authenticate result, verified credentials, and proven process retirement. No provider pins, contracts, timeout bounds, retries, or public APIs change. Existing documentation already specifies these sign-in semantics; no documentation change is needed. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
45a3a92e6d |
fix(environments): persist sandbox ownership before creation (#18357)
## Thinking Path > - Paperclip manages companies and the agents that do their work. > - Remote execution creates provider resources that need a durable owner. > - The server could create a sandbox before it saved its lease. > - A failed lease write could leave cleanup data only on the old web server disk. > - This change saves a company-scoped operation before creation and transfers ownership atomically. > - Recovery finds the original Daytona resource or records an explicit manual reconciliation outcome. ## Linked Issues or Issue Description **What happened?** A sandbox could exist without a lease after a database failure or company rebind. Failed cleanup recording could then create a local fallback file. Replacing that web server could lose the cleanup reference. **Expected behavior** A failed admission write makes no provider call. An interrupted creation retains a committed operation. Recovery must not create a second sandbox or destroy an attached lease. **Steps to reproduce** 1. Run the disposable PostgreSQL cases in `server/src/__tests__/sandbox-creation-operations.test.ts`. 2. Reject the lease insert after the controlled provider creates its resource. 3. Start a new receiver process with a different empty web disk. It must find the same resource and create one cleanup owner. Related ownership report: Refs #14672. This change addresses new allocations before lease persistence. It does not replace existing lease recovery. ## What Changed - Add an idempotent migration and a small creation-operation table. Retain original company, environment and run identities across deletion. - Commit creation intent before run, ad-hoc lease, and plugin task replacement calls. Fence duplicate requests and ambiguous lease commits. - Retain a known returned task resource ID in the existing operation before attachment. Preserve manual reconciliation and never delete an ambiguously attached resource. - Keep company and run authority checks and atomic lease transfer with the lease owner. - Reuse Daytona names, labels, read-only inspection and claimed cleanup. Keep provider payload handling in the provider. - Retain nonsecret credential identity hashes. Refuse recovery lookup under a changed credential value. - Record manual reconciliation for missing/deleted or explicitly inactive saved credentials, versions, and provider vaults. Keep temporary resolution failures retryable. Never reactivate a disabled credential. - Use transaction signals and concrete deadlines. Failed records back off independently. Manual reconciliation stays durable and quiet. - Remove new orphan-file fallback writes from the covered paths. Keep legacy file replay and its tests for cutover. - Replace obsolete fallback-producer assertions with normal-entrypoint durability tests. Keep cleanup, diagnostics and warm handoff coverage. ## Verification - [Independent current-head verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534744320) passed 190/190 combined tests, 40/40 incoming TestOps tests, full workspace typecheck, and build. PR head `958af986130f25e136065ad30d1fdd843c46255c` has tree `e6c3e5b02182d5941a70ccf5bf55638df3c2e28d`, identical to tested combined commit `b9d7a9c2e9b30f0153685fb24a5f70b56132201f`. Real PostgreSQL, compiled Rust Runner and controlled object providers were used; external-S3 integration remains outside this local proof. - Current head `958af986130f25e136065ad30d1fdd843c46255c` normally merges master `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57`. The sole conflict was the maintenance test: a parameterized case now preserves both legacy-local `local_storage` and valid shared-S3 `provider_mismatch` checks, including the parent local receipt, exact version identity, missing-snapshot/no-seed refusal, and unchanged persisted row. The three master TestOps files remain intact. No production, schema, migration, or package changes were made after the fully qualified prior head `0b4d05505e`. Independent review approved the resolution; the full managed-skill-content file and Cloud guard controls pass 58/58 (105.11 seconds). [Required PR CI passed on this exact merge head, first attempt](https://github.com/paperclipai/paperclip/actions/runs/41136987412): 54 successful checks and three intended skips. The additional Paperclip review is neutral at 4/5; two external Superagent checks (Contributor trust and Security Scan) remain in progress with zero annotations and no assessment, so they are not passing scan evidence. Greptile reviewed exact head `958af986130f25e136065ad30d1fdd843c46255c` at 5/5; the final thread audit has zero unresolved threads. All completed evidence below retains its recorded revision scope. The additional Paperclip Review on `958af98613` is neutral at 4/5 with a Ship verdict and no must-fix or should-fix item. Its optional observation is that a generic create rejection remains pending/manual even when an operator believes allocation did not occur. This change preserves uncertainty because the generic create-error contract does not prove provider absence; no new error classification or provider-specific compensation was added. - [Independent final-source verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6534328247) passed all 189 real-PostgreSQL/Runner and controlled-fixture cases, full workspace typecheck, and build. The integration commit `c8411a8528386f4824b340ec5fdaaed1efb9ea69` has the same source tree as final PR head `0b4d05505e` (tree `bda65efc14b4c904d0f589154db77bb160641c8c`). Controlled object-provider coverage passed; the separately guarded external-S3 suite was not run because the bounded local MinIO pull was unavailable. This includes the two final compatibility fixes. - The first final-head CI attempt exposed one E2E timing failure in `disconnect-write-recovery.spec.ts:181`: the title stayed unchanged through the 15-second session-recovery poll. The same original case passed in 13.8 seconds on immediately prior head `7ec06f77b7`, and all UI/session/E2E source is identical across those heads. The failure log and CI artifacts are retained; no cause is claimed. One failed-job-only retry is authorized on this evidence, without changing a timeout, assertion, or production behavior. [The single failed-job retry completed successfully](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2): all 35 E2E shard cases passed (6.6 minutes), including the original session-recovery case in 14.0 seconds. The initial failure remains disclosed and its cause remains unproven. - Final head `0b4d05505ee0aa72f2981cdff74761406960ec73` adds two independently reviewed integration fixes after the complete 7ec CI matrix exposed two deterministic failures. Server 9/12 failed legacy local-storage maintenance because the CLI reached the new Cloud server startup guard; server 5/12 found the Daytona package version did not match its bumped manifest. The failed run is retained at https://github.com/paperclipai/paperclip/actions/runs/41131204203. It is not passing evidence and was not blindly retried. - Maintenance now constructs the existing explicit storage dependency after loading the required persisted Cloud snapshot, and passes it to both maintenance owners. It can verify legacy local storage before provider cutover while normal Cloud server startup remains guarded. The original case reproduced red locally, then the complete managed-skill-content and managed-cloud-prerequisites files passed 57/57 (32.05 seconds), including missing-snapshot refusal, unchanged stored configuration, and existing server guard controls. Daytona package version now matches manifest 0.1.10; the complete bundled package identity file passed 14/14 (3.98 seconds), and explicit manifest/lifecycle controls passed 2/2 (328 intentionally filtered; 3.42 seconds). An initial workspace-filter command matched no project and is not counted as pass evidence. The final diff changes no sandbox runtime or schema behavior. [Full CI is green on this exact final head](https://github.com/paperclipai/paperclip/actions/runs/41133619710/attempts/2): 56 successful checks, three intended skips, one additional incomplete/neutral Paperclip Review, and no pending or failed checks. The neutral review is not a passing assessment. [Greptile reviewed the exact final head at 5/5](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6529396969), with no new actionable issues and all review threads resolved. - Final combined head `7ec06f77b7c14a745c569d18401f16df3d26b626` includes master managed-skills migration 0417 and shared-storage prerequisites from `198218bf55` through normal merges. Sandbox creation moves to generated `0418_curved_lady_mastermind.sql`; its idempotent SQL is byte-for-byte unchanged. The regenerated 0418 snapshot contains both sandbox creation operations and the master skill mutation receipts, and points to the master 0417 snapshot. Master SQL and journal history remain intact. Exactly five snapshots remain, 0414 through 0418. The only sandbox-specific changes since the approved credential policy are migration metadata/number and the matching test/document references; the sandbox runtime files are unchanged. - [Independent final combined verification](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6533960026) passed on integration revision `e6f05b4cf0cd`, containing final sandbox head `7ec06f77b7` and actual master `198218bf55`: migration/declaration checks, all 158 real-PostgreSQL/Runner and controlled-fixture cases (251 seconds), full workspace typecheck (81 seconds), and build (35 seconds). - Final combined migration proof: fresh real PostgreSQL has both tables. A separate original-0417 sandbox table with a retained pending row survives two applications of the renamed 0418 SQL unchanged, with the expected primary/index and foreign-key structure. Migration numbering/safety checks pass. The normal runtime migration reapply case passes 1/1 (20 intentionally filtered cases; 46.69 seconds). The first temporary qualification harness stopped before any database assertion because its external `.ts` file selected CommonJS; the `.mts` harness corrected that setup and passed. Independent review approved the exact metadata and reference diff. Earlier full suite and live-provider results below retain their stated revision scope. - Credential-policy head `c606d473b03709f27577a55a1dfc0b0d3b4359fd` handles the remaining inactive-credential review warning. The existing boundary accepts only concrete secret-owner 422 codes and exact no-details disabled/coming-soon vault messages. Unknown 422, authorization errors, provider wrappers, and temporary database/provider failures still propagate. Independent source review approved this three-file correction. No generic resolver, schema, provider protocol, retry cap, or activation behavior changed. - Creation/cleanup qualification at `c606d473b0`: all 39 cases pass against real PostgreSQL (81.27 seconds). The new cases use actual disabled/archived saved secrets, a valid disabled version, and provider vaults created ready before normal API updates disable them or mark them coming soon. Each remains manual through restart/repeated scans, with zero provider lookup and no retry deadline. The independent plugin idle blocker remains intact. The temporary 503 control retains retry ownership and later recovers one cleanup owner. Before this correction, all five inactive-state cases remained pending; that failed evidence is retained. The prior 18-case known-ID task proof below belongs to `237144fcf6`, whose helper and task path are unchanged by this correction. - Known-ID head `237144fcf69a249c0e0310563428c0f55ed6d55c` also addresses the independent task-replacement reliability warning. The existing creation operation now saves only the known nonsecret returned resource ID before expiry validation and attachment. The write is scoped to operation/company/original environment and allows only a missing or matching ID. It changes no ownership state and copies no resolved configuration, credentials, or provider payload. Independent source review approved this four-file fix. - The complete real-PostgreSQL task suite passes 18/18 (9.52 seconds). Rejected attachments retain the ID through manual reconciliation and repeated scans. A real committed attachment followed by an injected lost response retains the same active lease and attached operation, with no release or second create. Same-ID retries work; conflicting IDs and foreign companies cannot change it. The initial three regression cases failed because the ID was absent, before the fix. - The shipped Daytona manifest does not declare task support, so it cannot enter this generic replacement path. The dispatcher requires both declared task support and an environmentTask worker method. Generic drivers without creation lookup still use manual reconciliation, with a known returned ID retained when the database write can commit. No automatic lookup capability or unconditional compensation was added. - Previous boundary-fix head `8ba77def151ea85eeae2db7693254db7f87673b0` fixes the fresh review finding about missing credentials. Only known secret-owner 404 results at the recovery secret-resolution boundary select the existing manual outcome. Provider inspection and transient database/provider errors remain retryable. Independent source review approved this change. - Secret-boundary focused acceptance at `8ba77def15`: all 34 cases pass against isolated real PostgreSQL. The complete creation suite passes 16/16 (100.75 seconds); the complete cleanup coordinator suite passes 18/18 (99.83 seconds). The new cases use normal saved-secret creation and deletion, confirm zero provider lookup, restart/repeated scans, no automatic deadline, preserved independent plugin idle blocking, and no additional idle blocker after the controlled worker supplies its quiet proof. A temporary 503 resolution failure preserves pending ownership and later recovers one cleanup owner. No additional live provider resource was created. - Failed attempts remain part of the evidence. The initial red run reproduced the pending-operation bug and also exposed a numeric-deadline test harness mistake, which was corrected. A later combined run reported an existing replacement-receiver failure and a new idle-fixture assertion failure; it was interrupted while the next test file remained in imports. An isolated diagnostic confirmed that the manual state was correct but the installed fixture plugin correctly blocked sleep without its independent quiet proof. The corrected test retains that blocker assertion before supplying the controlled worker witness. The unchanged replacement-receiver case passes in the final complete 16-case run; its earlier failure is not counted as a pass or assigned a proven cause. No timeout or production idle guard was relaxed. - Greptile reviewed previous combined head `7ec06f77b7c14a745c569d18401f16df3d26b626` at 5/5, with no unresolved review threads. Its CI produced the two concrete integration failures addressed above. The earlier CI and live evidence below remain historical evidence, not a claim that the current full gates have completed. - Master integration update: merged `dbff3747b8` and then latest master `76fde891cc` without rewriting branch history. Only `doc/DATABASE.md` conflicted; the sandbox ownership, object-backed run evidence, and cloud configuration sections all remain. The master merges changed no sandbox implementation, migration, or provider code. Migration numbering/safety checks pass and exactly five schema snapshots remain. At merge head `f5d23d4a59828835d01c150f83fa2645fb2d3bec`, the two focused creation/coordinator suites pass all 32 cases against isolated real PostgreSQL (64.57 seconds). Those merge-head checks precede the missing-credential fix below. The earlier live Daytona proof still covers the unchanged normal creation/lookup/teardown path; the new missing-credential branch is qualified with real PostgreSQL and controlled provider hooks. The optional Superagent Security Scan was inconclusive on prior head `7ec06f77b7`: its pinned PR merge-base checkout failed or timed out, with a neutral conclusion and zero annotations. An earlier scan could not start within 120 seconds; a bounded rerun request returned HTTP 404. These scanner failures are not passing security-scan evidence. On final head `0b4d05505e`, the additional Paperclip Review check is neutral: its assigned review execution did not complete and produced no validated assessment. Fresh Greptile 5/5 and independent source review are separate evidence. Previous-head required CI, independent review, and the live Daytona qualification below passed. - [Complete previous-head CI](https://github.com/paperclipai/paperclip/actions/runs/41105726052) passed at `0b5cbb763ab8faa25aa0ff1d8a405cdc0d3db409`: 53 successful checks and three intended skips, including typecheck, full tests, build and E2E. Greptile is 5/5 on this head and both original findings are resolved. - Initial head: 200 PostgreSQL runtime, creation and coordinator tests passed. The corrected head also passes all 32 creation and coordinator tests, including deletion before/during lookup, a failed lookup after deletion, restart, settled polling and actual idle-safety checks. Cases cover failed admission, failed lease writes, concurrent recovery, lost provider replies, ambiguous transfer retries, company rebind, environment deletion, stopped runs, lookup backoff, restart, drain and settled SQL quiet. - Separate receiver processes: passed. A controlled HTTP provider retains the resource while three independent receiver processes use two empty web disks. Recovery creates one cleanup owner and never repeats creation. - Fresh PostgreSQL migration and two additional applications: passed. The pending creation row survives unchanged. - Server TypeScript and SDK/shared build checks pass after the corrections. Migration numbering/safety checks passed. Exactly five schema snapshots remain. - Daytona adapter: all 330 tests passed, including stable-name lookup, ownership checks, deletion and the updated manifest version. An earlier run exposed the version expectation and it was corrected. Previous-head CI passed. - Live Daytona acceptance passed on `0b5cbb763a`: isolated real PostgreSQL, normal runtime acquisition through independent receiver processes, actual Daytona plugin/SDK 0.203.0, one disposable Ubuntu 24.04 sandbox (1 CPU / 1 GiB RAM / 1 GiB disk). A forced lease insert failure retained the committed operation; a fresh receiver with empty disk found the same sandbox, repeated recovery retained one cleanup owner, and the actual claimed cleanup sweep expired the lease and resolved the operation. Independent SDK lookup returned typed not-found; ordinary recorded-ID inspection returned destroyed. This successful attempt made one create, one recovery lookup, and one teardown. Two earlier failed harness/provider-confirmation attempts are disclosed in the [live acceptance record](https://github.com/paperclipai/paperclip/pull/18357#issuecomment-6530313054). The coordinator independently queried all three exact resource IDs through the SDK and confirmed typed not-found for each. - Complete CI-affected heartbeat/plugin warm-task suites passed 21/21 after test corrections. An unchanged workspace exposure case passed in isolation after the prior CI shard reported an unhandled readiness rejection. Complete previous-head CI and refreshed review passed. ## Risks - Deleted companies retain an explicit manual reconciliation outcome. Recovery does not retry missing credentials or insert a lease for a deleted company. - Only Daytona declares automatic creation lookup. Other uncertain providers retain `needs_reconciliation`. A missing creation name alone cannot prove that a late allocation will never appear. - Pending operations use the existing process-local signal and startup recovery model. This does not add cross-replica wake delivery or prove deployed sleep. - Legacy direct provider probes and custom-image setup use separate lifecycles. They are outside this replacement of the run-lease orphan-file fallback. The normal Test path that acquires a runtime lease is covered. - Existing orphan files still require the separate cutover inventory and settlement. This change does not delete them. - The schema change is additive. The new SQL can be reapplied. Historical SQL and journal entries are preserved; only the oldest generated snapshot is pruned. ## Model Used OpenAI `gpt-6.1-sol`, with tool use, code execution and repository review. The harness did not report the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge The required branch status checks (`ci / verify`, `ci / e2e`) and all test/build jobs passed on this head. The non-required Superagent Contributor trust and Security Scan checks are still running without findings; they are not claimed as passes. No required check or branch rule is overridden. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
81ecaf0654 |
fix(server): bound failed lifecycle verification (#18383)
Failed lifecycle verification could allocate another test environment every minute indefinitely. Count failed host verification tests separately from claims, persist a repair wait for confirmed configuration failures or three generic failed tests, and clear it through authorized retry or execution configuration changes. Preserve typed temporary execution-target failures through adapter checks so timeouts, transport failures and interrupted probes keep automatic recovery. Retain existing readiness, authorization, termination and cleanup gates. Document the policy and add real PostgreSQL persistence and actual OpenCode adapter regressions. Verified 398 focused tests (one Windows-only test skipped on macOS), repository-wide typecheck and build, structural checks, fully green current-head CI and Greptile 5/5 with no unresolved comments. Co-Authored-By: Paperclip <noreply@paperclip.ing>canary/v2026.1030.0-canary.19 |
||
|
DottaandPaperclip
|
8595efc20b |
fix(evals): support official TestOps job metadata (#18385)
## Thinking Path > - Paperclip manages AI agents and verifies real work through Product E2E tests. > - The fixed Chat feature workflow uses TestOps to select and report one backlog case. > - Official registration exposes a workflow-query URL and a Branch parameter. > - The existing reader accepts only workflow-file or numeric URLs, and the admin launcher requires an operation parameter. > - These assumptions reject valid official metadata before the feature can run. > - This pull request supports the exact observed form while preserving source, selection, and once-only execution checks. ## Linked Issues or Issue Description Refs: #18322, #18375. **What happened?** The second zero-model bootstrap stopped with a generic job-mismatch code. Ordinary TestOps UI inspection then showed an official workflow-query URL outside the accepted literals and a Branch-only job. The exact first failed predicate in that bootstrap remains unknown. **Expected behavior** Accept that exact fixed-workflow URL. Preserve the existing master Branch value when the job has no operation parameter. Keep rejecting foreign URLs, unknown parameters, wrong branches, and conflicting operation IDs before mutation. **Steps to reproduce** Pass the official query URL and Branch-only metadata through the existing bootstrap, selection, and admin-launch owners. The new pure controls exercise those actual helpers without a CLI, backend, or model call. **Paperclip version or commit** This change starts from `0e15c92073a09e50df99190a37f69a86286b989b` after #18375. **Deployment mode** GitHub Actions and the protected TestOps reporting environment. ## What Changed - Share one exact URL predicate across bootstrap, evaluation selection, and admin preparation. Add only the observed official workflow-query literal. - Support Branch-only, operation-only, and Branch-plus-operation jobs with strict definition IDs, names, values, and duplicate checks. - Preserve the exact master Branch value. Set an existing operation parameter to `evaluate`; otherwise use the fixed workflow's existing default. - Use the same checked parameter vector for job and plan writes and their readbacks. - Report distinct fixed bootstrap predicate codes without response values. - Document when the protected operation-parameter ID is required. - Align one existing Cloud-maintenance test with the inherited managed-Cloud S3 policy. Preserve the same-version local-storage proof in the parent, use valid synthetic Cloud settings in the child, and check the expected provider mismatch without S3 requests. Retain the no-seed and unchanged persisted-row assertions. ## Verification - URL controls: three failures on the parent; all pass with the correction. - Branch controls: two failures and one correct rejection on the predecessor; all pass with the correction. - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs`: 40 owning pure/mock controls pass on Node 26.4.0. - Two MJS syntax checks and `git diff --check` pass. TypeScript noEmit was not run for this MJS/documentation change. - The first [normal CI run](https://github.com/paperclipai/paperclip/actions/runs/41131094974) failed only in the managed-skill Cloud fixture and its dependent verification gate: 46 jobs passed, two failed, and one skipped. The tested merge included a new upstream policy that rejects the test's local-disk Cloud configuration. The exact caught runtime exception remains unknown. - Three pure controls using the actual policy validator pass: the old invalid Cloud fixture rejects, the new synthetic S3 fixture admits, and an invalid master key rejects. An initial tsx wrapper IPC permission error executed zero controls and remains a separate setup failure. - The PostgreSQL/maintenance-child test was not run locally. On new-head AWS CI, `managed-skill-content.test.ts` passed all 17 tests in server shard 9. - [Normal CI](https://github.com/paperclipai/paperclip/actions/runs/41133876636) is green on exact head `b76d5dd40d7711f1b5f6b6316b1114c0aae02d57`: 48 jobs passed and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed this exact head at 5/5 with no actionable findings or unresolved review threads. - No actual CLI, backend, model, provider, app, database, build, or bootstrap was run to validate this correction locally. ## Risks The exact original bootstrap predicate is still unknown. This fixes a demonstrated metadata false negative and preserves the prior failed run. Actual backend parameter types and integration dispatch still need verification. Unknown metadata stops before the once marker and writes. This PR changes no workflow, credential, case, model, grader, clock, or private-evidence policy. It does not claim a passing feature result. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
198218bf55 |
fix(config): require shared storage for managed cloud (#18350)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed Cloud instances must retain shared objects and secrets across server replacement. > - Implicit storage defaults can select local disk or an unintended bucket. > - Cloud runtime settings now come from a canonical database snapshot. > - This pull request validates bootstrap prerequisites before database setup and effective S3 settings after snapshot resolution. > - The benefit is an actionable refusal before local signing state or HTTP setup, while valid canonical Cloud settings work. ## Linked Issues or Issue Description **What happened?** A managed Cloud instance could start with local disk storage, an implicit bucket, or a local key file. After canonical Cloud configuration landed, checking only environment or file values would also reject a valid database bucket. Shared schema defaults could make an omitted raw bucket appear explicit. **Expected behavior** Managed Cloud requires external PostgreSQL, a stable environment master key, and effective S3 settings with a nonblank bucket. The canonical snapshot remains the configuration authority. Invalid configuration refuses startup before signing or HTTP effects. Self-hosted defaults remain compatible. **Steps to reproduce** 1. Select managed Cloud with its tenant token or managed configuration document. 2. Leave storage as local disk, or omit the S3 bucket from a raw canonical snapshot. 3. Start the server. Startup must refuse. An explicit valid bucket in the canonical snapshot must work without a file or environment bucket. **Paperclip version or commit** Current base: `4edd10804e2b42941aa2038e90c362c13a0b98dc`. **Deployment mode** Managed Cloud. Ordinary self-hosted defaults remain unchanged. Related PRs: #18335 owns canonical Cloud runtime configuration. #18327 owns S3 run logs. #18352 owns credential persistence. #18347 owns cutover checks. This PR adds configuration prerequisites to those existing boundaries. ## What Changed - Validate external PostgreSQL and a stable environment master key at the Cloud bootstrap boundary, before database setup. - Validate effective S3 settings after canonical snapshot publication and before Cloud signing, runtime consumers, or HTTP setup. - Require a nonblank bucket in the Cloud snapshot schema. Reject omitted or blank buckets in every legacy Cloud file before one-time import normalization, even when an environment bucket is supplied. Keep shared self-hosted schema defaults. - Preserve typed bucket provenance at the existing file-read boundary. Accept explicit canonical bucket names, including `paperclip`, and environment settings when the snapshot is null. - Validate effective legacy storage with the existing resolver before seeding. A rejected candidate leaves no canonical rows, so a corrected source file can be imported on retry. Environment overlays are checked but never persisted. - Add real PostgreSQL snapshot and migration-retry tests, startup refusal tests, and environment/file compatibility controls. Document that Cloud always requires its stable environment key, even with an external default secrets provider. ## Verification - Latest focused acceptance passed 199/199 in six files (54.29 seconds), using the command below plus `server/src/__tests__/cloud-configuration-migration-postgres.test.ts`. Real PostgreSQL proves rejected storage leaves all three migration authority rows absent, then a corrected legacy file imports on retry. An external-default-secrets control proves the deliberate unconditional Cloud master-key contract. - Before the peer correction, the frozen product diff passed 178 tests in five files: managed-cloud-prerequisites, cloud-runtime-config, server-startup-feedback-export, config-file, and cloud-configuration-readers. Command: `pnpm exec vitest run server/src/__tests__/{managed-cloud-prerequisites,cloud-runtime-config,server-startup-feedback-export,config-file,cloud-configuration-readers}.test.ts --maxWorkers=1 --no-file-parallelism`. Real PostgreSQL used a qualified POSIX test-only PG18.6 override because the host cannot supply the stock runtime's SysV semaphore allocation. - Coverage includes raw database and legacy omitted/blank bucket rejection, valid database and environment buckets, null snapshots, explicit default-named buckets, self-hosted compatibility, and refusal before local signing mutation. - The real AWS SDK configured-provider control uses a simulated loopback S3 service. It writes, reads from a fresh provider, and deletes an object. It also reads an encrypted credential with the stable environment key. This does not qualify production cloud permissions. - `git diff --check` passed. Final merges add unrelated master changes from #18241, #18363, and #18370; the focused storage product diff is unchanged. - Peer review reproduced a fresh legacy local-disk import that gained the default bucket before an S3 environment overlay. The importer now requires a raw explicit bucket for every legacy Cloud file. Six controls cover omission/blank values with and without an environment bucket; the full focused rerun passed 184/184 across all five files in 194.89 seconds. - Initial revised startup testing passed 167/168: one fixture omitted the database URL and failed before its intended key assertion. The corrected final run passed 178/178. An initial direct server typecheck failed because Runner declarations were not built; this is a setup failure, not a compile pass. Integration verification rebuilds dependencies before full checks. - Prior head `fae1e4aa` passed complete CI with 53 successes, two intended skips, and Greptile 5/5. `de71345f379cecb71dbfb219f35e14f942c10a3f` then passed complete CI (55 successes, three workflow skips, and a neutral external review), with Greptile 5/5 and zero unresolved threads. The external review identified the preseed rejection problem; its correction is committed at `75e04d0d77e19dcef0a8cb070b6b518f0f173b14`. The final merge head is `19b809ea2823a6a4e5ceb48ef2c2bac4721733d6`; [Fresh complete CI](https://github.com/paperclipai/paperclip/actions/runs/41127309762) passed on that head: 55 successful checks, three workflow skips, and one neutral external review. Greptile reviewed this exact head at 5/5 with no actionable issues. Both earlier external review threads are resolved with the tested fixes. Earlier infrastructure retries remain documented in [the first diagnosis](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6529964690) and [the runner shutdown evidence](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6530364769). Earlier missing SDK build and fixture metadata failures were repaired before those passes. - Current frozen-head CI first attempt failed one unchanged Antigravity authentication success case in Runner Vitest 1/2 (`outcome=failure`, 1994 tests passed). The same file passed 11/11 locally in 2.33 seconds, and identical Runner source passed the prior complete CI run. Attempt 2 retried only failed jobs (the Runner shard and dependent aggregate). Both passed without a source change. No second retry was used. The exact timing cause is not established. [Failed job](https://github.com/paperclipai/paperclip/actions/runs/41127309762/job/124309181778). - No production data, buckets, or live credentials were used. Complete local repository suites are not duplicated in this worktree; current-head required CI supplies the full suite gate. ## Risks - Managed deployments with local disk, omitted buckets, file-only keys, or missing external database URLs now refuse startup. Operators must supply valid prerequisites before rollout. - Required deployment environment overlays must remain stable across replacements. For example, a saved local-disk provider with an explicit bucket can use an environment S3 provider; removing that required overlay correctly refuses startup until the environment or canonical configuration is corrected. Environment settings are never silently copied into the canonical row. - Effective legacy storage must also pass before first seeding. A bad candidate does not create authority rows. Existing saved rows remain authoritative and can require environment overrides or a canonical configuration change if they are invalid. - A present legacy Cloud file needs an explicit raw bucket even if its saved provider is local disk or its environment supplies a bucket. Correct the file before one-time import, or use the existing absent-file/null-snapshot environment-only setup. Environment overlays are not persisted as operator choices. - A previously stored canonical nonblank `paperclip` bucket is authoritative. Prior normalization erased whether that value was originally explicit. This change cannot recover that provenance and does not reject existing nonblank canonical values. - Validation proves configuration shape, not cloud permissions, database reachability, or key compatibility with existing encrypted data. Keep the master key stable across replacements. - Cloud snapshot JSON parsing is stricter; omitted or blank raw S3 bucket values need correction. No database migration or provider operation changes are added. These prerequisites do not prove that every server persistence path is disposable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`. Context window: unknown. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0e15c92073 |
feat(skills): make managed content independent of web-server disk (#18334)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use installed skills and exact version pins during execution. > - Managed skills still depended on mutable directories on the web server. > - A replacement server could lose acknowledged edits or remove an unassigned skill whose directory was absent. > - This pull request makes immutable versions the authority for managed content and installed repository reads. > - The server can rebuild delivery files from durable content after replacement. > - This establishes skills readiness for ephemeral web servers. General server draining and workspace recovery remain separate work. ## Linked Issues or Issue Description **Subsystem affected** Skills services, shared contracts, database receipts, Skill Studio, and runtime delivery. **Problem or motivation** A fresh web server must discover, read, edit, and deliver protected installed skills using the same database, object storage, secrets, and supported application artifacts. It must not need its predecessor's files or process memory. **Proposed solution** Use `managed` ownership with a null source locator. Read installed content by immutable version. Prepare packages before a short publication transaction. Persist execution selections before materialization. Recover exact pinned runtime assets and reject deletion while active or recovering pins exist. **Alternatives considered** Reconstructing a mutable installed directory preserves the old source-of-truth ambiguity. Permanent dual writes create two competing authorities. This change uses one managed model after a bounded legacy conversion. **Roadmap alignment** This improves the existing Skills Manager, Studio, version history, and Cloud execution capabilities. The maintainer supplied the implementation contract. This does not add a separate skills product or deploy fleet changes. Related work: #18082 supplies immutable package storage. #17273 addresses edits outside the API for linked local skills; this PR keeps that source category separate. #2648 is a closed earlier ephemeral-filesystem proposal. ## What Changed - Add managed ownership and focused content publication and execution materialization modules. Keep the archive format and package storage tables. - Publish managed create, save, delete, fork, rename, restore, and catalog content with version checks. Add durable create receipts. Preserve file-save receipt responses. - Return file version identities. Keep Studio drafts after a conflict and provide an explicit reload action. A delayed reload cannot overwrite newer typing or navigation. - Persist exact runtime selections, including empty selections. Rebuild pinned assets and verify their admitted digests on a replacement server. - Preserve source access on forks and disconnected copies. Keep the existing workspace policy category for managed skills. - Add bounded conversion, eligibility, and independent verification phases. Load persisted Cloud configuration in the standalone CLI without seeding or changing it. Reject partial current versions against installed inventories; recover only complete saved content or the exact source. Document storage migration blockers and the compatible rollback floor. - Add separate-process PostgreSQL/S3 qualification and update domain, retention, policy, and UI regressions. - Pass the plugin worker manager into Studio execution, admit restricted `skill_test` mode through native Runner contracts, and preserve admitted scope across native and REST tools. Native tests cannot create ordinary tasks, browse company data, or use external tools. - Select the exact durable Studio revision even when unassigned or Beta skills is off. Persist Studio output and audit atomically with native task completion, preserving the result on replay. ## Verification - Final revision `13e8af480b88ba6fea5737bac4bd81104d604792` passed full CI: all 49 workflow jobs completed successfully ([run 41127396098](https://github.com/paperclipai/paperclip/actions/runs/41127396098)). This includes typecheck, test suites, Runner checks, browser checks and build. Greptile review 15 is 5/5 on that same revision, with zero unresolved threads. The PR is ready for review. - Local workspace typecheck and build passed after integrating master. Focused checks passed: 33 execution/completion cases, 52 Runner HTTP cases, 16 managed-content cases, 17 Studio cases, the new real-process CLI regression, map checks and UI token gates. The CLI regression proves inspection does not seed missing Cloud configuration. Server typecheck passed after that fix. - The independent-process PostgreSQL/S3 matrix passed through normal HTTP skill routes. It covers A → B → C replacement, lost commit responses, upload crashes, concurrent saves, binary contents and executable delivery. Metadata listing and warm selected delivery retain zero object downloads. Local timing measurements are recorded; they do not establish hosted latency. - Hosted A → B → C passed on the owned disposable test tenant with external Postgres, S3 and a real remote Paperclip Computer. Predecessor temporary homes were absent. C read and executed B's acknowledged managed save. New runs used repository v1 while v2 was installed. Hashes matched an independently checked exact upstream commit. - Compatible image rollback to `6518b323` passed. Studio retained the unassigned fork's C save. A fresh remote run executed the newer managed content and historical repository pin. The actual shell event confirmed expected hashes, stdout, exit 0 and mode 0555. - Runtime candidate `9200e3fcb4` deployed successfully ([Cloud run 41124147876](https://github.com/paperclipai/paperclip-cloud/actions/runs/41124147876)). The final Studio test delivered only the unassigned fork's exact v3, executed its script and persisted a successful output snapshot. Independent command-event inspection confirmed all three hashes, stdout, exit 0 and mode 0555. All 14 installations were version-backed. Independent verification checked all 25 versions: 23 remote objects, two bundled database snapshots, zero blockers. - The deployed image is `9200e3fcb4`. Later master integration, the client-only delayed-reload draft guard and the CLI bootstrap have separate automated proof. The CLI bootstrap was also exercised against the tenant's stored configuration and storage through the same read-only services. This PR does not claim that the deployed image is the final PR revision. - Live Studio checks exposed worker wiring, admission, access scope, exact selection and completion-publication gaps. They are fixed. Failed attempts remain in the evidence. The full local suite exceeded 80 minutes under host contention and was stopped. Later combined/isolated local attempts also timed out; they are not counted as passes. Full CI covers the integrated suites. Earlier CI found an obsolete restart-scope fixture, which was corrected and passed all 52 HTTP cases. - Reproduction and migration: [`doc/skill-ephemeral-web-servers.md`](doc/skill-ephemeral-web-servers.md). Source revisions, deployments, versions, run IDs, hashes and failure history: [`doc/plans/2026-10-30-ephemeral-web-skills.md`](doc/plans/2026-10-30-ephemeral-web-skills.md). ## Risks - Legacy conversion requires stopped admission and retired filesystem-authoritative writers. Pre-conversion binaries are not supported rollback targets afterward. - Cloud readiness requires remote object storage. Local provider receipts and provider mismatches remain blockers until objects are copied and verified. - Linked local sources retain filesystem semantics and are outside the cloud guarantee. - Cold reads still fetch the existing archive format. Warm runtime preparation retains the verified-cache path. - Bundled reconciliation preserves an installed head. Existing release snapshots remain selectable. An older server must not replace a newer head with its shipped files. - No production volumes, fleet defaults, or general deployment coordination change in this PR. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and browser use. The exact backend model ID and context-window size are not exposed in this session. No subagents were used. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d9354bbfc3 |
Add a portable agent question and answer resumption journey (#18346)
## Thinking Path
> - Paperclip helps people manage agent work and keep human decisions
attached to tasks.
> - Product E2E journeys verify the browser, native agent, saved state,
and cleanup.
> - The classic suite already checks a choice question followed by a
text question.
> - Deployment-targeted journeys need the same behavior on an attached
instance.
> - Production questions can use provider input, the dedicated semantic
tool, or the authenticated native API fallback.
> - The basic journey must verify the original card and native
continuation for each supported path.
> - This pull request shares the scenario and browser actions, adds
strict API receipt evidence, and verifies the saved document after
reload.
> - Separate tool-documentation qualification and the original failed
measurements keep their existing scope and outcomes.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
It adds portable coverage for answering agent questions and resuming the
same task on an attached deployment.
**Current behavior**
The classic suite has the two-question scenario. The portable catalog
does not expose it. Its shared native-path grader recognizes provider
and dedicated semantic receipts, but rejects a valid completed native
API creation receipt.
**Proposed behavior**
Expose `question-answer-resume` as a deployment-targeted journey. Answer
one choice question and one text question in the browser. Require
server-owned creation evidence, unchanged forms, exact board answers,
bound continuation, an agent-authored task document, and readback after
reload.
**Reason and benefit**
The same story can qualify local and hosted targets through their normal
provider setup and cleanup. It catches lost answers, duplicate or
foreign work, premature output, and stale documents.
## What Changed
- Register the journey with a three-run maximum and positive model
allowance. Accept one to three attributed work runs at both collection
and retained-result boundaries.
- Share the existing two-question scenario, browser answer actions, and
behavioral grader. Keep separate tool-documentation grading unchanged.
- Recognize a completed native API creation receipt only when its HTTP
201 result retains the original pending card, exact
task/company/agent/source run, idempotency key, and full normalized
form. Require the receipt to remain unchanged after the answer and
through the final checkpoint. Missing, pending, foreign, or altered
proof still fails.
- Check signed-in board ownership, each native continuation, no
premature or extra output, document authorship, and the saved revision
after reload.
- Wait for the authenticated product region before screenshots. Recheck
origin and path after readiness. Bound readiness and capture by the
campaign deadline.
- Add closed stage-specific errors, catalog coverage, sanitized receipt
calibration, and documentation. Include the provider-form and
browser-question presentation helpers in the definition fingerprint.
## Verification
- Fresh QA staging qualification passed at
`7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks
passed, cleanup passed, and all three native runs used the configured QA
target and model. The [sanitized run
report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/)
records the measured pass. This is one successful qualification, not a
reliability claim.
- Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3`
contains the one main merge through
`fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of
the four-file qualified search fix
`4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity
matches. It preserves the project and search registrations and atomic
search readback. The question live qualification ran at the earlier SHA
above; the final integration head has separate local and CI
verification.
- Eighteen question journey, helper, and test file blobs are identical
between the qualified and integration SHAs. This includes clarification
flow and evidence, question grader and browser helper, the sanitized
receipt fixture, screenshot helper, provider-form helper, browser
question presentation, continuation scenario/readiness/flow, shared
agent validation, and user actions. The question run-count contract and
dispatch branch are also unchanged. Reviewers can compare those paths
directly between the two commits.
- All 622 selected Product E2E, catalog, question-resumption,
necessary-question, report, and readiness tests pass after the main
merge. After the exact search delta, all 145 narrow
question/search/project/contract tests and all nine real Chromium search
regressions pass. Map checks, combined credential-free preview, source
fingerprints, and the earlier workspace-link preflight pass. The local
TypeScript attempt reports stale Runner declaration errors for
`onAccountingEvent` in unchanged native-session code; fresh official CI
provides the clean-build type gate. The three search source/test blobs
match the qualified search merge, and the three project source blobs are
unchanged.
- An in-memory probe executes the actual launcher fingerprint
expression. Each project, search, and question implementation input
changes the digest. Separate red probes first reproduced the omitted
provider-form and browser presentation helpers; both pass after their
source inputs are added.
- The sanitized retained native API response-shape regression fails
before the grading correction and passes afterward. Negative controls
reject missing, pending, foreign, changed, spoofed, or wrongly bound
receipts, answers, continuation, model, environment, and document
evidence. All nine provider/semantic/API path combinations pass
calibration.
- The real local Chromium screenshot probe fails before the readiness
repair and passes afterward. Login redirects and sign-in pages at task
URLs remain excluded with no screenshot files.
- The two original attached failures remain retained: the first stopped
at early screenshot capture; the second stopped before answering at the
former API provenance gate. An earlier sandbox run also retained one
report subprocess failure; the full selected tests then passed with
local subprocess IPC access.
- The prior integration CI had an unchanged canary mock `EADDRINUSE`
failure. That history is retained; its documented retry is owned by the
coordinator. No application API, schema, production UI, model, runtime
policy, credential, or deployment configuration change is included.
Final integration head
|
||
|
DottaandPaperclip
|
56957f2bd9 |
fix(ci): isolate test selection and require current CI CODEOWNERS (#18376)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - PR CI selects affected suites and test files to reduce feedback time. > - A selected test inherited the parent selection when it launched an independent runner fixture. > - Ordinary source filenames containing config or setup also forced broad coverage. > - AWS runner access must depend on current CI CODEOWNERS, not only a stored numeric roster. > - This pull request fixes both selection problems and verifies owners before AWS allocation. > - Contributors retain GitHub-hosted CI. Authorized maintainers retain AWS test and final verification jobs. ## Linked Issues or Issue Description Refs #18362 and #18348. The selected server run for #18369 failed in `vitest-chat-shards.test.ts` because its temporary fixture inherited `PAPERCLIP_CI_TEST_SELECTION` and attempted `git rev-parse HEAD` outside a Git repository. #18366 could not use graph selection because `session-config-metadata.test.ts` matched the broad filename rule. The AWS setting was already enabled. The reviewed outsider PRs correctly used hosted runners; a maintainer PR used AWS, including final verification. This change strengthens that boundary rather than enabling AWS for outsiders. ## What Changed - Remove selection settings from Vitest child environments after the wrapper validates the selected shard. - Match conventional Vite/Vitest configuration filenames rather than any occurrence of config or setup. Keep graph tracking for actual setup and configuration dependencies. - Bind rerun actor lookup to the exact run attempt. Require the author, event sender and rerun actor to be current CI CODEOWNERS and members of the existing numeric allowlist. Read the resolver and CODEOWNERS from one protected master revision. - Run metadata authorization on GitHub-hosted capacity before any AWS allocation. Keep authorized test and final verification placement on the AWS fleet. - Add a real selected-server regression that launches nested chat fixtures, negative authorization cases, and qualification for a real config-named test. - Update the developer and profiling guides. - Retire eight legacy PR workflow SHA authorizations and one deleted qualification-branch reference from the live `paperclip-public-pr` runner group. The remaining three references are the trusted PR workflow and the existing full-stack/protocol evaluation workflows on master. No active PR run referenced the retired versions. ## Verification - Selection regression tests: 13 passed, including the real nested chat runner. - Routing, planner and profiling tests: 100 passed, including 69 routing cases. - `actionlint` passed for both changed workflows. `git diff --check` passed. - Confirmed protected master still requires CODEOWNER review. A live lookup found four CI owners, with three of six numeric roster entries eligible and three denied by the new owner check. - `pnpm -r typecheck` and `pnpm build` passed. The full local Vitest suite was stopped after repeated embedded PostgreSQL initialization failures; it did not pass. On final revision `0b40ae15b7996603e405edd4157a5c337e0405c7`, both Linux qualification jobs and all required CI gates passed. The main PR workflow completed with 48 successful jobs and one intentional skip, including final verification. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41126881716). Greptile reviewed the final revision at 5/5 with no inline findings. - Shard and workflow contracts: 44 passed. CI first caught an outdated assertion that required the routing job to use AWS; it now requires hosted authorization, and the policy job passes. - A read-only live gate exercise preserved the existing merge-freshness denial when master advanced beyond the available merge ref. It allocated no runner and was not a successful admission proof. ## Risks - The short metadata authorization job now depends on GitHub-hosted queue capacity. This prevents a stale roster from allocating AWS before current CODEOWNERS are checked. Authorized test and final verification jobs still use AWS. - Identity/API failures fall back to hosted CI. No caller can override the trusted owner list with PR contents. - Configuration dependencies still require conservative graph handling. Existing setup dependency, opaque I/O, stale manifest and wrong-lane tests remain in place. - The runner group is restricted to selected repositories and selected workflows. Its workflow allowlist was verified and narrowed through the GitHub settings UI. Legacy pinned callers must update to the current trusted workflow before using AWS; active PR runs used master references. The current PR workflow authorization and evaluation workflow authorizations remain in place. ## Model Used - OpenAI Codex, based on GPT-6. The exact serving variant and context limit are not exposed in this session. Used reasoning, repository inspection, code execution and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — targeted suites passed; the full local suite hit the PostgreSQL startup limitation documented above, and full Linux CI passed - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
a1845b3b64 |
fix(evals): capture TestOps registration after watch (#18375)
## Thinking Path > - Paperclip manages AI agents and verifies their work through Product E2E tests. > - The fixed Chat feature workflow prepares a TestOps case and plan before any model run. > - Its first zero-result bootstrap completed, but the watched child did not receive the generated job-run ID. > - The official watch log contained a job-run ID, so the missing child variable did not prove that registration was absent. > - This pull request reads official context after watch and validates the linked backend records. > - The workflow retains closed registration metadata and stops on unknown or foreign context. ## Linked Issues or Issue Description Refs: #18322. **What happened?** The successful zero-result bootstrap retained `official_job_run_id_missing` from its watched child. It could not prove the backend job and workflow binding needed for the next setup step. **Expected behavior** After successful watch, read the generated context, validate its current GitHub run and fixed workflow, and retain only closed IDs and parameter metadata. **Steps to reproduce** Run the fixed workflow's `bootstrap` operation on master. Inspect its setup artifact after watch completes. No evaluation or synthetic passing result is needed. **Paperclip version or commit** The original bootstrap used merged #18322. This follow-up starts from `76fde891ccc29283b3f5c8dcdce089c5b3d937d1`. **Deployment mode** GitHub Actions metadata-only setup in the protected reporting environment. ## What Changed - Run an internal context reader after successful bootstrap watch in the same protected step. - Parse only numeric official job-run and launch IDs in memory. Do not log or save raw command output. - Validate current-run, project, launch, job, workflow, and bounded parameter metadata before writing an exclusive registration receipt. - Retain the two known registration receipt paths with the existing setup artifact. - Preserve the original inside-watch receipt, case reconciliation, provider isolation, and evaluation limits. ## Verification - `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs tests/runner-e2e/testops-chat-reporter.test.mjs`: 48 pure/mock/FS controls pass (36 prior and 12 new). The actual local Node version was 26.4.0. Normal Node 24 CI remains required. - Four MJS syntax checks and `git diff --check` pass. No TypeScript noEmit claim is made for this MJS-only helper change. - Actionlint passes for the changed workflow. - Normal [GitHub Actions CI](https://github.com/paperclipai/paperclip/actions/runs/41123894542) passes on exact head `58396de10f0c49294b0bbc0f4709c29990d57d53`: 48 jobs succeeded and one skipped. Both required `ci / e2e` and `ci / verify` pass. - Greptile reviewed that exact head at 5/5 with no actionable findings or unresolved review threads. - No local CLI, provider, model, app, database, or build qualification was run. No replacement bootstrap or paid evaluation was dispatched. ## Risks The official context or backend DTO shape can differ from the supported format. The reader then stops with a closed failure receipt. Pure tests do not qualify the remote CLI or backend. The original zero-result bootstrap and its missing child-context observation remain part of the history. The case's actual outcome remains unmeasured. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool use, and pure test review. The runtime does not expose the exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
6174b4d7c8 |
fix(runner): allow session reset after confirmed task retirement (#18373)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native warm sessions retain a Runner owner between turns. > - Closing that owner must stop execution and preserve valid recovery state. > - A failed suspension can leave a rejected close promise after the task has stopped. > - A session reset then retries that same promise and blocks every fresh session. > - This PR separates task retirement proof from checkpoint reuse for a new session. > - A reset can proceed after exact stop proof, completed handoff, and file collection. ## Linked Issues or Issue Description Refs #18011. That change preserves checkpoint authority during warm preparation. This change handles a stopped owner whose checkpoint cannot be reused. Related: #14695 covers managed files retained with warm processes. A search found no duplicate stopped-owner reset fix. **What happened?** A warm Runner can stop and complete its task handoff while close reports `runner did not durably suspend before checkpoint`. The transport caches that rejected close promise. Core retains the warm owner. Each session reset retries the same promise before it can launch a provider. A controller restart clears the in-memory owner and permits recovery. **Expected behavior** A reset to a new normalized session should release the old owner when its exact task has proved that execution stopped and its handoff completed. Core must collect stopped instruction files before it admits the new session. An unconfirmed owner must continue to block replacement. **Steps to reproduce** 1. Complete a warm native run through a plugin task provider. 2. Change configuration so Core retires the warm owner. 3. Let the task stop and complete, but fail durable suspension proof. Close keeps its rejected promise. 4. Reset the session and start another run on the same task environment. 5. Observe the old close error before any new provider work starts. ## What Changed - Keep a lease-bound retirement receipt in the plugin task owner. It requires `executionStopped: true` and successful task completion after credential handoff. - Accept an unrecoverable suspension error only during retirement for a different normalized session, with that exact owner's receipt and settled Core tool results. - Collect stopped instruction files before releasing the warm owner. Preserve old checkpoint evidence. - Keep same-session, active-owner, incomplete-handoff, unrelated-owner, unresolved Core tool results, and other close-error fences. - Report Core tool settlement on the close error after ingress retirement and drain. Default missing proof to false. Expose the error through the server's Runner shim. Add regression tests and document the recovery boundary. ## Verification - The reset regression fails with the new handling disabled and passes with it enabled. - Both affected suites passed: 1,032 tests. The final 15 focused owner/reset cases pass after adding the tool-settlement guard. - The assembled regression uses the real task-owner and backend binding. Its controlled task provider proves stop and completion. Close returns the same rejected promise on every call. - Negative cases cover incomplete handoff, other close errors, same-session continuation, provider change with the same session ID, unrelated receipts, unresolved Core tool results, and failed file collection. Collection can later succeed without replaying stop or reading the old checkpoint. - Five real-process transport cases pass. They cover an insufficient suspension deadline, the cached rejected promise, late tool completion within and beyond the deadline, interruption, and result-persistence failure. Unsettled callbacks produce `semanticToolsSettled: false`. - `pnpm -r typecheck` passed with the repository's pinned Rust toolchain on PATH. After the final contract change, Runner TypeScript build and server `tsc --noEmit` passed. - The local full-suite run was stopped at the requester's direction. Full regression and build coverage are delegated to GitHub CI. - Full GitHub CI passed on `ef0040d80af679ce6d4a130468e15f75678c1f4c`. Standard Greptile reviewed the same commit: 5/5, zero comments. Apex then reviewed that same commit: 5/5, zero comments. Final checks confirm no unresolved review threads and no merge conflicts. - Initial full CI on `00c002fd5a` failed one scheduled-retry route test at 18/20 cancellations. Controlled cancellation latency reproduced the short poll-budget failure. Before the next cycle, master advanced with #18322, which coordinates that test at the first completed batch. The separate local timeout commit was dropped during rebase. The upstream test passes locally. The rebased run passed 46 targeted owner/reset and scheduled-retry route cases. This PR has no scheduled-retry source or test diff. - No deployed provider run or tenant rollout is claimed by these source tests. ## Risks The exception applies only to a new normalized session and an exact completed plugin task. The old checkpoint remains evidence and cannot authorize reuse through this path. The original suspension failure can remain visible on its run. This fix prevents repeated startup failures after a reset; it does not hide unconfirmed cleanup or make every suspension succeed. Missing task receipts and failed file collection continue to block replacement. No schema migration, plugin protocol change, Rust Runner binary change, or new polling is required. Tests use a controlled task boundary; they do not certify every external provider implementation. ## Model Used OpenAI Codex, GPT-6 family, with tool use and local code execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal issue or instance details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] Greptile Apex is 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
4edd10804e |
fix(evals): batch task search result readback across renders (#18370)
## Thinking Path
> - Paperclip lets people manage AI agents and saved work.
> - Product journeys check user actions against saved state.
> - The task search journey compares the exact visible task identities.
> - A search can replace rows while the harness reads an old indexed
locator.
> - The attribute read can then time out after the correct results
appear.
> - This pull request reads the visible links together and preserves
strict result checks.
## Linked Issues or Issue Description
Refs #18355.
**What happened?**
An attached Cloud attempt passed the task search journey. A later
attempt on the same harness and app revision failed at search-result
readback. Its owned screenshot showed the correct target and completed
control. Authentication, collection, and cleanup passed. The failure
remains recorded and the journey has not been admitted to the daily
plan.
The reader called `all()`, then awaited `isVisible()` and
`getAttribute()` for each indexed locator. When a search replaced the
old collection between those reads, the next attribute read waited for
an index that no longer existed.
**Expected behavior**
Read the matching visible links together. Compare the complete identity
set against the owned target and controls. A render transition must not
consume the whole poll deadline on a stale row index.
**Steps to reproduce**
1. Render three task links in the task collection.
2. Let the reader observe the third link as visible.
3. Replace the collection with the two correct search matches before its
attribute read.
4. The old reader times out waiting for `nth(2)`, even though the two
correct rows are present.
## What Changed
- Read visible link attributes in one batch. Discard a detached snapshot
and retry within the existing poll deadline.
- Keep the exact identity comparison, duplicate detection, control
exclusion, saved-content checks, and zero-run checks.
- Add a real Chromium render-transition regression and invalid-result
cases. Extend the stateful flow test for a detached snapshot.
- Document the readback boundary. No app UI, target configuration, or
cleanup behavior changes.
## Verification
- Review follow-up: the existing journey wait helper is now exercised
with the real Playwright poll. Its first read evaluates the actual
reader callback on detached browser DOM nodes and returns no evidence.
The second read contains an extra owned control. Only the third, exact
pair succeeds. The test observes all three reads. The 30-second deadline
and grading behavior are unchanged.
- Rebased onto current master, including the merged project journey.
Credential-free discovery confirms exactly one `project-task-context`
and one `task-search-filter` entry.
- Latest rebased checks passed: all nine Chromium search regressions,
all 381 Product journey unit tests, runner typecheck, discovery, maps,
and diff checks. Exact head
|
||
|
Nicky LeachandPaperclip
|
b056026407 |
fix(ui): keep task plugin views available for manual access (#18363)
## Thinking Path > - Paperclip helps people manage agents and their tasks. > - The task side panel can display a plugin's agent view. > - Users need that view even while a task is idle. > - Activity currently controls both manual discovery and automatic opening. > - This pull request keeps manual access available and uses activity to request automatic opening. > - The same authorized agent viewer serves both entry points. ## Linked Issues or Issue Description **What happened?** A task view disappeared when its run ended. Users could not select it until the current run reported matching activity and its readiness check passed. This also excluded declared manual views without an activity mapping. **Expected behavior** The task launcher offers declared views for its running or assigned agent. Selecting one uses the same viewer as the agent detail page. Matching activity and readiness open the pane and select the view by default, once per run. **Steps to reproduce** Open an idle task assigned to an agent with an installed task-panel contribution. Open the panel launcher. The view is currently missing. Related work: #18131 and #18239. ## What Changed - Return manual panel declarations independently of run activity and readiness. - Mark panels that should open automatically after matching activity and readiness. Keep existing run attribution, retry bounds, and dismissal behavior. - Keep an open viewer mounted when a run ends. Focus an already open manual view when activity starts. - Preserve saved manual tabs while plugin and initial run discovery load; remove contributions after discovery confirms they are absent. - Share the run-scoped dismissal key between the task page and panel controller so dismissed views do not reveal an empty pane. - Use the current running agent for declared manual views that have no activity mapping. - Update the plugin authoring contract and feature recipe. ## Verification - Two new regressions failed before the fix: idle discovery returned no view, and a manual contribution opened automatically. - 105 focused hook and panel tests pass. They cover idle/manual access, declarations without mappings, completion, saved-tab restoration, automatic focus, activity attribution, retry bounds, dismissal, and missing/disabled plugins. - Thirteen task-page panel integration tests pass. New regressions reproduce and fix a dismissed manual view revealing the outer pane. They cover desktop/mobile and both workspace modes, and verify that a later run can still open the view automatically. - `node scripts/check-token-gates.mjs`, `node scripts/check-maps.mjs`, and `git diff --check` pass. - Local UI typecheck failed with the reused dependency cache: workspace modules/exports and generated Storybook catalog files are missing or stale. It reported no errors in the changed files. The final head passed the clean GitHub typecheck. No broad local build or repository test run was attempted. - The preceding head passed [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/41116028688), with 48 jobs succeeding and one skipped. Its final review scored 5/5 and identified a narrow saved-view restoration race. This revision adds the missing initial-run loading guard. Two delayed-response regressions failed before the correction and now pass; they cover a different assignee and an unassigned task. - The final head `bbde3a28f7ac0390da8f3fe81eb9abd2f3020695` passed [GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/41119772438) and the additional [full CI run](https://github.com/paperclipai/paperclip/actions/runs/41121827023), each with 48 successful jobs and one skipped. Standard Greptile scored 5/5 on this head; both review threads are resolved. The small final loading correction used the permitted exception to another final-review cycle. - The additional full CI run initially failed the canary bundle install because npm advertised a newly published `tslib@2.9.1` archive before its tarball was available (HTTP 404). After the archive became available, one retry of the failed canary and dependent aggregate passed. No dependency or lockfile change was needed. - Hosted acceptance of this follow-up requires merge and deployment; the earlier hosted activation checks cover the previously merged behavior. ## Risks Manual views can now open while idle or before readiness. Each viewer retains its normal authorization and status handling. Declarations without an activity mapping now appear in the launcher. Automatic opening still requires authenticated activity and the authorized readiness result. Saved tabs wait for plugin discovery before removal. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, terminal tools, browser verification, and code execution. The exact serving variant and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
fd008979f5 |
test: wait for the evidence expiry worker to settle (#18241)
## Thinking Path > - Paperclip coordinates AI work and retains execution evidence. > - Evidence retention uses an asynchronous worker and a shared scheduler. > - The retention test checked the next deadline as soon as an archive disappeared. > - File removal can finish before the worker publishes that deadline. > - This pull request waits for worker settlement before asserting its next wake. > - The test keeps its deletion, busy-session preservation, and quiet-worker checks. ## Linked Issues or Issue Description **What happened?** CI for #18239 failed in `native-harness-backup-expiration.test.ts`. The healthy-expiry test observed the file removal while `nextWakeAt()` was still null. Subtracting the current time then produced a large negative number. **Expected behavior** The retention test should check the deadline after the asynchronous worker has settled. **Steps to reproduce** Run the server test shard with a slow filesystem completion. The deletion predicate can become true before scheduler publication finishes. ## What Changed - Keep the existing file-removal wait open until tracked work is idle and the expected busy-session retry deadline is registered. - Keep the held archive and quiet-worker assertions. ## Verification - Original CI failure: [server shard 3/12](https://github.com/paperclipai/paperclip/actions/runs/40976031169/job/123804262266). - The targeted test passes with its real PostgreSQL fixture: `node node_modules/vitest/vitest.mjs run --project @paperclipai/server server/src/__tests__/native-harness-backup-expiration.test.ts -t 'lets healthy saved expiry preempt' --maxWorkers 1`. - After rebasing onto master, the focused retention test passes again with its real PostgreSQL fixture. The PR now changes one test, with seven insertions and two deletions. - Master independently fixed the authentication timeout and GitHub-recovery checkpoint race. Their redundant commits were dropped during rebase. - [Full GitHub CI](https://github.com/paperclipai/paperclip/actions/runs/40984842413) passes on `bbaa7c3c384f6c220ce2c979f0e9b5d01ddb1f77`: all 48 jobs succeeded. Standard Greptile is 5/5 on that head, and GitHub reports the PR clean. - Broader coverage runs in GitHub CI. No local full build or full test suite was run for this assertion-only change. ## Risks Low risk. The retention test waits for the condition it already asserts. The existing deletion, busy-session preservation, future-deadline, and quiet-worker checks remain in place. No production code or time budget changes. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, terminal tools, and code execution. The serving variant and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1030.0-canary.18 |
||
|
DottaandPaperclip
|
76fde891cc |
Add bounded recovery file inventory before server cutover (#18347)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Accounting, credential recovery, and sandbox cleanup can retain legacy files on an instance volume. > - Replacing that volume while files remain can lose unresolved work. > - There is no verified production backlog, so an empty installation needs no migration. > - The recovery owners must decide whether each record is valid and settled. > - This pull request adds a bounded file inventory and an optional cutover gate. > - The benefit is a safe check that preserves obligations without reading secret material. ## Linked Issues or Issue Description **Problem or motivation** Operators need to check the actual affected instance directories before replacing a web server. A skipped corrupt file or failed read must not appear clear. **Proposed solution** Add a Node script that inspects only the three legacy recovery directories for an explicitly chosen instance. Default mode reports counts. The optional check mode blocks when files or uncertainty remain. **Alternatives considered** A bulk migration system has no established backlog to migrate. Reimplementing the owner parsers could expose credentials and still would not prove settlement. This check never reads file bytes. **Roadmap alignment** This is a small operational prerequisite for safe cloud server replacement. It does not add a recovery backend or change resource ownership. Related recovery-owner changes: #18354 (accounting), #18352 (credentials), and #18357 (sandbox ownership). This inventory remains independently usable with existing files. ## What Changed - Add a pure classification policy and a bounded filesystem boundary. - Add `node scripts/recovery-cutover.mjs --instance-root /absolute/instance` with optional `--check`, directory override, and entry budget. - Report observed lower-bound counts and safe status summaries. Do not print paths, filenames, identifiers, raw errors, or file contents. - Add one dependency-free CLI acceptance suite and a Vitest bridge so the same cases run in normal server CI. - Document existing owner entry points, override selection, stopped producers, and retained-volume requirements. ## Verification - `node --test scripts/recovery-cutover.test.mjs`: 9 cases pass. The suite starts the actual CLI with synthetic legacy directories. - Pending receipts, nested credential files, orphan records, malformed bytes, temporary files, unreadable files, unsafe paths, symlinks, read failures, bounds, and configured overrides all block safely. - Empty or absent recovery directories clear. Source file bytes remain unchanged. Report-only mode and check-mode exit codes are tested. - `pnpm exec vitest run server/src/__tests__/recovery-cutover.test.ts --pool=forks --maxWorkers=1`: the normal CI bridge passes and runs the same 9 cases. - Cached frozen dependency installation, script syntax checks, and `git diff --check` pass. The lockfile is unchanged. - Independent source review found the read-only boundary sound. - [Complete current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41097614666) passed at `bf5cc7f067041c10cd0294ff26756678da943976`: 53 successful checks and two intended skips, including typecheck, full tests, build, and E2E. Greptile is 5/5 with no review threads. - One failed-job retry was needed for an existing heartbeat timing failure in server shard 4. The relevant heartbeat source and tests match the base branch. Local diagnostic runs had 61 pass/2 fail and 7 pass/1 fail, with changing timing failures; neither run is claimed as green. The bounded retry and all failed evidence are [recorded here](https://github.com/paperclipai/paperclip/pull/18347#issuecomment-6529629804). No heartbeat code or test was changed. ## Risks - The scan is a point-in-time inventory. Stop producers for the final gate. It is not an atomic drain certificate or protection against hostile path mutation. - All files remain unverified. The check cannot certify validity or Postgres settlement. Existing owners retain that responsibility. - The scan does not inspect other instances, volumes, Runner delivery, or database obligations. It does not establish that production cutover is ready. - No migration, deletion, decryption, or replay command is added. Symlink paths are rejected, including system aliases; use canonical absolute paths. ## Model Used - OpenAI `gpt-6.1-sol`, with reasoning, tool use, and code execution. Context window size was not supplied. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
f53db0c70f |
perf(server): overlap OpenCode setup and trace reply recovery (#18367)
Overlap eligible OpenCode GitHub and assigned MCP preparation. Trace deferred reply discovery and recovery, and forward the native GitHub transport recorder. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
e54354d9d3 |
fix(ai-connections): save quota credentials without recovery files (#18352)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed AI connections keep the selected account in the secret store. > - Server quota refresh can receive a replacement token before its database save commits. > - That path currently writes an encrypted recovery file on the web server. > - Cloud web replacement should not need that file to use a saved account. > - This pull request saves the replacement directly and retries the same value in memory. > - The benefit is a smaller cloud storage contract with an explicit sign-in recovery limit. ## Linked Issues or Issue Description **What existing behavior does this improve?** Server-owned OpenAI quota refresh and credential persistence during web replacement. **Current behavior** Quota refresh checks local encrypted recovery storage before token exchange. It writes the replacement to that storage before the secret transaction commits. A replacement web receiver can therefore need the old web volume. **Proposed behavior** Required database reads and locks must succeed before exchange. The same logical refresh saves its returned replacement directly, with at most three save attempts. It checks the current secret after an uncertain commit response. A process that loses the unsaved replacement can require sign-in again. **Reason and benefit** Postgres remains authoritative. This removes new quota recovery files without adding another credential vault. Existing files remain readable for cutover. Local-only Google retirement keeps its existing behavior. Remote Google subscription sessions remain unsupported. Related public work: Refs #14056 for managed Codex credential handover. Searched open and closed credential-refresh pull requests; this change does not duplicate their provider login work. ## What Changed - Keep the exchanged replacement and its source generation in the current refresh operation. Save retries do not repeat OAuth exchange. - Reconcile a lost commit response from the saved credential. Grant changes, secret versions, session epochs, selectors, and revocation prevent a stale save. - Stop creating or preparing quota recovery files. Preserve existing record settlement and local Google compatibility. - Document the v1 sign-in limit and existing Runner persistence-before-completion contract. - Add PostgreSQL and controlled-provider regressions for save rollback, lost replies, revoked or changed authority, receiver process loss, and pending Runner credential handover. ## Verification - [Complete current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41099132607) passed at `e4ed6551a707c814ddb2395a7358106b1e04857a`: 53 successful checks and two intended skips. This includes full typecheck, tests, build, and E2E gates. - `pnpm -r typecheck` and `pnpm build` passed at `1ebe75984be9bf19d518728d4ff440a2d7f3dbce`. Later branch changes only repair tests. Rebase adds an unrelated master UI performance fix; the credential and Runner handover source did not change. - Focused credential tests passed: 2 files, 160 tests (145.62s). Command: `pnpm exec vitest run server/src/__tests__/ai-connections.test.ts server/src/__tests__/plugin-task-credential-writeback.test.ts --maxWorkers=1 --no-file-parallelism`. - `pnpm test:run` was deliberately interrupted after 17m41s in its first general-server chat integration phase, before any suite result. The controller exited 130 and its wrapper exited 1. This is an incomplete local run, not a full local pass or an observed test failure. The coordinator stopped duplicate long local work; the complete current-head CI suite subsequently passed. All owned test and PostgreSQL processes were confirmed stopped. - Database tests use real PostgreSQL 18.6 with POSIX semaphores through a test-only loader. Provider requests use controlled fixtures; no production OAuth mutation was made. - Greptile reviewed current head `e4ed6551a707c814ddb2395a7358106b1e04857a` at 5/5 with zero comments. - The captured focused-test logs contain none of the replacement-token fixture values. - The receiver-loss test kills a separate Node process after one HTTP token exchange while PostgreSQL blocks its secret insert. A new receiver then uses the existing sign-in flow. - The Runner case rejects a real deferred secret commit. Provider completion remains uncalled until a new receiver commits the retained credential. - Initial setup failed because this host exhausted System V semaphore resources. Another setup attempt lacked built SDK entries. Both were setup failures, not passes. After setup repair, the first real run passed 159/160 tests; the remaining test asserted the retired directory-fsync contract. The replacement test forbids recovery writes and requires one committed exchange without a file. ## Risks - If all three saves fail and the producing server process exits, the unsaved replacement is lost. The next failed use can require sign-in again. This is the accepted v1 limit. - Existing unresolved recovery files still require inventory and settlement before their old volume is removed. This PR does not perform deployment or delete those records. - Local Google subscription retirement still needs its existing local recovery ownership. It is outside the supported remote cloud scope. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, repository tools, code execution, and controlled integration tests. The context window size was not supplied. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9f524ec1a7 |
fix: persist native Runner usage before transport acknowledgement (#18354)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent execution must preserve usage so Paperclip can price work and enforce budgets. > - Remote Runner execution can outlive its web receiver. > - The receiver previously acknowledged Runner events before its later usage recorder saved the accounting receipt. > - This pull request saves required native usage receipts before that transport acknowledgement. > - A replacement receiver can accept the same retained report and create one ledger charge. ## Linked Issues or Issue Description **What happened?** Native remote Runner usage could leave its durable outbox before the web receiver saved its accounting receipt. The later recorder required a local receipt directory. A replacement web receiver could lose the only remaining report when the database write failed. **Expected behavior** The Runner retains unaccepted structured usage. Postgres owns accepted receipts. Repeated delivery preserves one receipt for each source report and one charge for the run. Missing usage keeps accounting pending. **Steps to reproduce** 1. Start a supported built-in native remote Runner. 2. Make the receipt database insert fail after the provider reports usage. 3. Replace the web receiver and its local directory. 4. Reconnect the same Runner and inspect the saved receipt and ledger. **Paperclip version or commit** The initial investigation used `ebd05e9f4c88c66af7473db86bbaacfd1544aa0a`. **Deployment mode** A web server with a remote native Runner and shared Postgres. Related work: #18327 owns output logs and provider traces. This change has no dependency on its S3 export path. #18031 covers interrupted Codex receipt recovery. #18005 covers legacy spool reads. #17516 covers Codex stop-boundary retention. ## What Changed - Add native producer receipt delivery through the existing Runner callback before transport acknowledgement. - Reuse `run_usage_receipts`, stable source sequences, and deterministic report identities. Reject conflicting replay and retry a lost commit response with the same receipt. - Restore per-turn usage from structured receipt facts. Preserve provider turns that share one PRP envelope turn. - Bind retained warm transport callbacks to the current run authority epoch, allowing old attachment control acknowledgements while preserving accounting ownership. - Persist Codex session baselines and cumulative totals in the existing provider checkpoint. Mark qualified whole-run deltas, preserve them across resume/restart/attachment, and leave old unmarked reports incomplete and unpriced. - Normalize the bounded final usage tail before the terminal event. Retain later increases for explicit reconciliation, block further attachment, and fail an affected successor without changing its baseline. - Complete runs from the accepted immutable receipt and its exact saved rate-card amount; preserve existing live-budget enforcement. - Skip the local receipt recorder for supported built-in native remote Runner execution. Keep local, legacy, and external Dot/Muse behavior unchanged. - Keep pricing, reservations, ledger settlement, source fencing, and budget holds with their existing owners. - Put the actual Runner integration test in the existing Rust-cached native test lane. Build before opening Postgres and give cold compilation its own bounded budget. ## Verification - Fresh Greptile review is 5/5 on `e7376a50c8162770da1d581ecf25c8cffac64520`, with no unresolved findings. Parent and independent peer review approved the final changes. - Real PostgreSQL 18.6 and a real Rust Runner exercise failed receipt insertion, receiver-directory loss, replay from the same producer, and exactly one 25-cent ledger charge. The actual accounting spool path stays absent. Optional logs/traces are disabled. The provider is controlled; no paid request is made and the fixture never pre-saves the expected receipt. - Producer checks passed 32 usage/provider tests,37 terminal controls, corrected completion, all 3 previously failing exit cases, and both full transport goal-continuation cases. Rejected active-goal completion retains its 120-token tail before failure. Baseline 100→successor150→33 duplicate old snapshots→old120 retains explicit incomplete reconciliation evidence and positive budget exposure. - Heartbeat executor checks passed 982/982. PostgreSQL tests preserve exact 1.002 USD pricing, the 100.2-cent ledger amount, budget holds, source takeover, partial/final receipts, cost-only corrections, equal rate-card replay, and stale-completion interleaving. Newer incomplete usage remains pending rather than settling an older snapshot. - Runtime replay and strict usage-schema checks passed 2/2; generated contracts and Rust formatting pass. Test-partition contracts passed 32/32. - [Final combined-head evidence](https://github.com/paperclipai/paperclip/pull/18354#issuecomment-6531526158) records full workspace typecheck/build and 41/41 accounting tests against all five cloud PR heads, including this final accounting commit. It includes commands and evidence limits. - Exact-head CI is green: 53successful checks, 6intentional skips, and the legacy Snyk status passed. Typecheck, build, Runner, server, browser, and aggregate gates pass. Runner Vitest1/2 initially failed an unchanged mocked Antigravity login fixture; its focused case passed locally, then one unchanged job retry passed 131 files/1995 tests, including all 11 Antigravity tests. The initial failed attempt remains recorded in CI. Initial local attempts exposed the host SysV semaphore limit and corrected harness path, IPC readiness, and queue-counter defects. A temporary test loader selected an existing POSIX-semaphore PostgreSQL 18.6 build without changing repository dependencies. An optional expanded debug provider run was interrupted at its unrelated 8192-turn rollover stress case; no full-file local pass is claimed. Required failure cases were run directly, and the full Runner CI remains required. ## Risks - The guarantee covers built-in native remote Runner execution. Local native execution, legacy adapters, external Dot/Muse execution, and server-originated model calls retain their existing contracts. - This does not provide traffic switching or general web-server failover. The existing controller identity, checkpoint, and generation gates still apply. - The receipt JSON gains optional structured producer facts. No schema migration or accounting S3 spool is added. Text output and raw trace payloads are excluded. - Codex accounting authority requires a Runner artifact built from this provider baseline implementation and qualified by digest. Update receivers and strict usage-schema readers before releasing that Runner; PRP does not negotiate this payload field. Existing unmarked Codex reports remain pending/unpriced. No deployed Runner artifact is claimed fixed. - Usage arriving beyond the bounded terminal tail may require manual reconciliation or a fresh session. Its evidence remains in the existing producer checkpoint/outbox; no adjustment ledger or recurring scan is added. ## Model Used OpenAI `gpt-6.1-sol` assisted with the implementation, tests, and documentation through code execution and repository tools. The context window size is unavailable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
dbff3747b8 |
feat(server): persist run logs and provider traces in S3 (#18327)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators use run logs and restricted provider traces to inspect agent work. > - Local log and trace files depend on the web server's persistent volume. > - That dependency prevents overlapping volume-backed Railway deployments. > - This PR reuses the instance S3 store for batched log and trace payloads with conditional publication. > - Existing Postgres rows retain their contracts; native export recovery reuses events already committed before Runner acknowledgement. > - S3-configured servers automatically copy and verify retained history before a later deployment cutover. ## Linked Issues or Issue Description **Subsystem affected** Server run logs, restricted provider traces, storage providers, and migration tooling. **Problem or motivation** Replacing the web server and its disk must preserve committed log output and retained traces. The current S3 mirror can repeatedly upload a growing file. Moving those payloads into Postgres would add database writes, WAL, replication, and backup work. **Proposed solution** Reuse the existing instance storage configuration, S3-compatible provider and credential chain. Store new run logs under run-logs/ and opted-in raw traces under provider-traces/, below the existing prefix. Cloud keeps its existing bucket and bucket-scoped credentials per stack. Add no new configuration switches. Store immutable payload segments and bounded binary range indexes directly in S3. Publish each batch with an ETag-conditional head. Keep existing database ownership and lifecycle records. Recover canonical native exports from their existing validated control event source, checkpointing export progress in S3. Add no new payload writes to Postgres. Preserve local defaults and migrate settled history with verified byte counts and digests. **Alternatives considered** Postgres payload chunks add control-plane load. Repeated whole-file mirrors amplify transfer work. The first index prototype left superseded index nodes; the final binary index keeps all normally published nodes reachable. **Roadmap alignment** This is a storage prerequisite for replacing the main web/API server. It does not enable deployment overlap, detach a volume, or change native Runner protocol acknowledgements. The maintainer approved this implementation scope. ROADMAP.md has no duplicate implementation item. Related PR: #16770 improves the existing in-flight mirror. This PR preserves that legacy backend and uses the existing instance S3 profile for new journals. ## What Changed - Automatically select S3 segments for new logs and opted-in traces when instance file storage uses S3. Reuse bucket, endpoint, region, prefix, path style and credentials. Remove separate backend switches. Keep local-disk instances local. - Share the existing storage configuration resolver with product files. Preserve config-file settings, environment precedence and the existing root prefix. Keep pre-existing run-log mirror settings for legacy history. - Add bounded buffering, conditional ownership fencing, immutable range reads, stable byte offsets, and completion hashing. - Dispatch all existing log consumers through the recorded backend. Coalesce output progress writes and publish live log messages after storage receipts. - Recover interrupted canonical native exports from their existing durable source. Exclude separate credential, cleanup, and reprojection records. Replay missing callback ranges and deduplicate late delivery. Repair settled archives after controller lease expiry. Bound shared repair work and stop it when all readers cancel. Serve the published prefix when best-effort repair fails. - Add encrypted optional trace channels, replacement reads, durable deletion tombstones, and deadline-driven local/external expiry cleanup independently of heartbeat scheduling. Stream historical source and remote verification. Propagate worker cancellation through trace open, conditional-write probing, upload, restore and reads. Fence deletion even before the first object head exists; abort and drain owned captures. - Retry transient storage errors, distinguish 409 from failed preconditions, and backpressure bounded log admission. Track interrupted trace imports explicitly so ordinary local deletion stays independent of S3. - Automatically migrate retained history through the existing delivery coordinator. Process one log and one trace per batch. Retry unresolved history with backoff and stop scans after completion. Keep the inventory/import command as optional diagnosis and repair. Verify source and remote bytes before switching references. Verify finalized native size and digest before setting its export cursor. Require ordered source coverage when completeness metadata is missing. Preserve original trace expiry. Block active, missing, corrupt, and unproven sources. - Preserve identity and run-secret redaction in live and recovered native logs. Register runtime log redaction secrets once through the existing encrypted registry. Keep agent private keys in their existing owner. - Add storage, migration, retention, UTF-8, and route-privacy coverage, plus operating documentation and qualification tooling. - Add no schema, payload table, per-record SQL, production dependency, or lockfile change. ## Verification - Final fixes at `b3a44c898e` passed all 116 selected startup, retention, attribution and cancellation controls. The disabled-scheduler cleanup and concurrent-repair defects reproduced red before the fixes. A timed-out repair still prevents later storage reads. - The merged source passed 124 storage and real-Postgres Cloud configuration tests. These include automatic history migration, existing environment/file vault keys, source-directory removal, five real-SDK stalled-request cancellation cases, shared capability-probe cancellation and multi-chunk trace ordering. Saved Cloud S3 settings work after the legacy config file is gone. - Workspace typecheck and full build passed at the merged parent `19f702d780`. Server typecheck and build passed again at `b3a44c898e`. - Actual app acceptance: finish a process-agent run through the normal API, stage its retained local-file input, then restart. Automatic migration reports one migrated item and zero blockers. Remove the original web home and restart at `b3a44c898e` with an empty home, the existing S3 profile and the existing vault key file. The normal log API preserves all five producer lines and exactly 788 bytes, with SHA-256 `74d9ae469da28bfe89b9b50a0e132aec24fe9e6d686496ed5e4ffa6167151036`. No log-specific storage settings are present. - Actual expiry acceptance: with heartbeat scheduling disabled, a trace remains unexpired after startup and later reaches its fixture deadline. The worker marks the row deleted and tombstones both S3 channel heads. The retained run log stays byte-identical. The process fixture emits no provider protocol frames, so nonempty encrypted payloads are qualified separately through the real SDK round trips. - Isolation tests use identical run identities in two stack buckets. Cleanup in one existing prefix leaves another prefix, another company and product files intact. Read-only Cloud source inspection confirms a bucket and bucket-scoped credentials per stack. Four existing Cloud storage environment-contract tests passed. No Cloud provisioning changed and no new live vendor permission probe was performed. - Earlier qualification passed 1,014 affected storage/native tests and 27 privacy-route tests with isolated Docker Postgres. An abrupt process test commits 1,000 records, receives receipts, dies by SIGKILL and loses its directory; another process reads every byte and continues. The real Rust Runner experiment acknowledges 32 records while the export cursor is zero, then continues the same Runner PID after controller loss. All 64 records export once with exactly 64 existing control rows. Repeated snapshot/Stop commands completed in 162/81 ms. - Emulator workload: 1,000 burst records used two batch PUTs. Eight steady streams produced 240 records with 104 conditional PUTs and 72 GETs over 6.9 seconds. These are synthetic operation counts, not fleet sizing or provider prices. - Failure evidence is retained. Compact native events over 64 KiB, missing file-key support, disabled-scheduler retention and repair failure handling all reproduced before their fixes. Fixture repairs addressed cold module setup, localhost/IPC permissions, the S3 bucket-name limit and a mode-0700 crash-test directory. Full local `pnpm test:run` was stopped after host PostgreSQL semaphore exhaustion; affected suites used isolated Docker databases instead. Native package library symlinks were restored with the package's standard hydration helper. - The `19f702d780` browser session-recovery case timed out while saving a task title. All eight browser shards passed on `b3a44c898e`, including that case, with no UI behavior or assertion change. Fresh-head CI passed at `b3a44c898e`: 55 successful checks, three skipped checks and one neutral check. The branch is conflict-free. - Map and module-boundary checks passed. Current-head Greptile is 5/5 with zero unresolved review threads. Reproduce the opt-in storage suites with `PAPERCLIP_LOG_TEST_S3_ENDPOINT` and a dedicated `PAPERCLIP_RUN_EVIDENCE_MIGRATION_DATABASE_URL` ending in `_test`. See `doc/run-evidence-storage.md` for operating limits, automatic migration, optional diagnostics and rollout qualification. ## Risks - Keep the existing per-stack bucket, prefix and vault key stable across replacements. Relocating that storage is a separate whole-instance operation. - This release automatically changes new log storage for existing S3-configured instances. Local-disk instances remain local. Canonical native source replay is qualified with a deterministic real Runner; actual remote execution and production-provider qualification remain rollout gates. Generic server-only diagnostics and optional raw trace tails without a surviving source can be incomplete after a crash. This does not establish full zero-downtime deployment. - Before deploying this release, qualify the actual S3 provider. It must enforce conditional writes and pass read, version, deletion, workload, and latency qualification. A capability probe rejects providers which ignore preconditions. Moto is emulator evidence only. - Dirty output is bounded and unconfirmed until S3 publication. Storage failure leaves persistence incomplete while task results and usage can settle. - Trace expiry is not yet represented by Cloud's external wake contract. Outstanding trace deadlines conservatively block sleep. - Automatic history copying requires the original source volume. Keep it until all retained resources pass verified import and clean-filesystem reads. Stop old binaries from creating new local-file runs. Missing or corrupt history remains blocked and visible in server logs. Rollback requires a compatible reader against the same external state. Ordinary object deletion does not erase retained historical object versions or backups. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with reasoning, tool use, code execution, and browser verification. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
6dcf4160dd |
perf(chat): load recent messages and page history on scroll (#18358)
## Thinking Path > - Paperclip lets people manage work done by agents. > - A standing agent chat can hold months of conversation. > - The chat route forced full-history mode, bypassing compact task loading. > - A cold visit fetched lifetime run summaries and hydrated historical transcripts. > - This PR starts with 20 recent messages and loads earlier pages on upward scroll. > - It preserves the reader's position, current questions, and draft while history grows. ## Linked Issues or Issue Description **What happened?** Standing agent chats forced full-history loading. A long conversation fetched full activity and run history before it became readable. Earlier comments also used an explicit load control. **Expected behavior** Open a small recent conversation quickly. Load older messages when the reader scrolls upward. Show an inline loading state. Keep the visible message and composer mounted through paging, live updates, and Back navigation. **Steps to reproduce** 1. Open a standing agent chat with hundreds of messages and completed runs. 2. Use a fresh browser query cache. 3. Observe full activity and run reads plus transcript requests for historical runs. 4. Compare the recent window and upward paging after this change. Related work: #18312 and #18323 restored compact task and collection loading. #14848 added reopening of visited agent chats. A GitHub search found no open PR for this chat-history optimization. ## What Changed - Return 20 contiguous recent comments for standing chats, with a stable oldest cursor. Keep the existing compact task exchange contract. Use the existing company/issue/time index for cursor reads. - Keep chats compact even if their task view previously expanded history. Retain observed messages through live catch-up and cache-backed return visits. - Keep fetched history pages, acknowledged sends, and the observed recent cursor separate. Reconcile live replies without losing history after a send or deleted cursor. - Hydrate metadata only for newly revealed message/run IDs; preserve prior batches while paging. Slow-send recovery refreshes the recent window. Receipts failures have an independent retry. - Hydrate run summaries only for loaded messages and the latest turn. Validate batches of at most 100 run IDs and keep the existing issue/company access rules. - Reuse hydrated terminal native streams when only history membership changes. Preserve retry, resume, mode changes, reconnect recovery, and interrupted recovery reads. - Load older messages, decision receipts, and task handoffs through upward scrolling. Support wheel, touch, and keyboard input. Keep pending questions visible. - Reserve a status slot while loading. Preserve a logical message anchor and the mounted composer. Pause failed page reads until an inline Retry. - Add database, route, controller, and scroll regressions, plus an opt-in production-build browser rig for long chats on desktop and mobile. Update design and feature/access documentation. ## Verification - The real PostgreSQL cold-open suite passes all 12 tests, including contiguous pages with equal timestamps, complete history, company denial, and scoped run IDs. - Activity route and OpenAPI suites pass 53 tests. The route run allowed local cold-import overhead; CI retains the repository's normal timeouts. - Three new paging regressions fail against the unchanged source. The candidate's focused paging/window suite passes 26 tests. Broader UI verification is recorded below when complete. - Desktop and mobile browser acceptance covers recent-only startup, per-frame reveal geometry, delayed and failed older pages, stable reading anchors and drafts, Back, older comment links, and one persisted live reply. - Full workspace typecheck and build passed locally. The final UI source also passes its typecheck and production build. - Affected UI suites pass: 212 controller/history/paging tests, 352 thread/scroller tests, 26 native-stream tests, plus metadata batching and live-cache hydration coverage. The issue mutation/access route suite passes 179 tests. - All 22 final desktop/mobile browser journeys pass, including delayed image growth. The isolated second-page request proof also passes six repeated desktop/mobile runs. A separate 6,000-message, 3,000-run fixture passes cold-open and delayed-paging checks on both screen sizes, still opening 20 messages and loading one 50-message page. - Final matched production timings use baseline `aa58fa1073` (`index-BaRiMbxd.js`) and candidate UI source `f9a26b9783` (`index-DPWTbsFF.js`, identical application source at final head `bf71ffd5af`). Chromium 151.0.7922.34, 1440×900, 4× CPU throttle, no added network latency, browser HTTP cache disabled, fresh application cache for each cold visit. Both variants block service workers and use the same warm local API server and immutable 600-message/300-run fixture. Only document HTML is intercepted for both variants; API requests are real. The common API revision is `109981f8c1`; subsequent commits change only UI and tests. | Usable conversation | Baseline median (range) | Candidate median (range) | | --- | --- | --- | | Cold direct URL | 6.57 s (5.69–7.45) | 3.07 s (2.50–3.28) | | Warm in-app return | 3.76 s (3.54–4.76) | 0.67 s (0.58–1.46) | | Back return | 2.53 s (2.12–4.20) | 0.51 s (0.47–1.18) | Five pairs alternate order. Every paired difference favors the candidate for all three journeys. Readiness requires a revealed, fully visible latest reply above the composer and an editable composer. Return timing starts after the Tasks route renders. Cold API requests fall from 953–954 to 71; API wire bytes fall from 517–559 KB to about 49 KB. Separate CPU profiles and request waterfalls confirm the removal of lifetime event hydration and are excluded from the five timing pairs. Both variants have the same expected missing optional plan document (404) and no browser exceptions. Control recent-window API reads have medians of 37 ms / 32 ms, with shared-host outliers up to 376 ms. The 6,000-message recent-window SQL uses `issue_comments_company_issue_created_at_idx` (22 index rows read for a 21-row probe). Geometry assertions retain a 2-pixel tolerance for the reading anchor and first-reveal layout. These are local synthetic measurements, not deployed staging latency. Earlier timing probes are excluded because they did not isolate service workers or composer clearance. Final head `bf71ffd5afb5684ba9105a3127f617e7adf5139c` has 55 successful checks, three optional skips and one neutral result, with no failures or pending checks. Greptile is 5/5 with no open review findings. ### Failure record Early local attempts exposed an unconditional scroller wrapper that changed the old task layout, and automatic retries on chat page failures. Both were corrected. The original task scroller structure stays in place. Metadata request assertions wait for initial connection recovery to settle, then verify only the new page’s reads; retry/resume/reconnect revalidation remains intentional. The initial review caught a send-before-scroll cache defect; its controller regression failed before the fix. Later rapid-paging checks caught scroll readiness being gated by metadata after the first reveal; readiness now stays enabled once revealed. Live replies use the separate acknowledged-message cache, then catch up from the server cursor. Visual review exposed a pagination wrapper bypassing existing composer clearance. The body is again a direct scroller child; cold-open browser checks require the full latest reply above the composer. A resize-observer regression failed before the fix and now passes. Controlled delayed inline images prove both pinned following and a stable reading anchor on desktop/mobile. The first performance fixture lacked valid native event envelopes and caused missing-log fallbacks. It is excluded from latency claims. The final fixture uses saved native assistant events and conversation session generations. The live test uses a temporary fixture-agent key, attributes the reply to an existing synthetic run, and revokes the key. It starts no model or agent process. Local database attempts failed during a Docker shutdown. The retained disposable database was restored and its suite passed. Early fixture errors included a missing conversation identity field and session generation. Mobile test setup also selected an offscreen sidebar link and reused queued human input. Optional announcements could intercept input; the fixture now dismisses them before the journeys. The scroll observer now waits for the delivered wheel event and settled animation frames, including when the viewport is already at the top. A concurrently running live test briefly changed another probe’s row count; the final acceptance and timing runs were sequential on separate fixtures. The final rig uses the visible mobile navigation and a saved agent reply. Anchor observation waits for the user's scroll to settle before measuring the effect of live updates. The full local `pnpm test:run` attempt hit repeated embedded PostgreSQL initialization failures and failures in untouched native-session executor suites; it was stopped after the repeated environment failures. The targeted real-PostgreSQL suite passed, and final-head CI is required before handoff. The first CI run also caught an exact mock expectation missing the added company scope; the expectation is updated and its 179-test suite passes. The final-head first CI attempt also stalled an assigned child task in the existing plan-approval scenario, then hit a heartbeat-run row lock timeout during fixture cleanup. The failed shard and aggregate check passed on an unchanged rerun of the same commit. Scenario assertions and timeouts were unchanged. The failure and rerun are retained in [CI run 41116120430](https://github.com/paperclipai/paperclip/actions/runs/41116120430). ## Risks - Historical transcript diagnostics no longer load on a cold chat open. Saved messages and conversation events remain available; diagnostic run views keep their full records. - Cursor paging must not omit comments with equal timestamps. Database tests cover timestamp/UUID ordering and the complete set. - Prepended or late content can move the reader. Browser checks cover retained row identity, viewport offsets, mounted drafts, and Back. Previously loaded pages stay in the query cache for its existing lifetime. - Local synthetic evidence does not establish deployed staging latency. This PR does not merge or deploy the change. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected checks; limits are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9e1735ec17 |
test(evals): add a fixed TestOps AWS chat feature route (#18322)
## Thinking Path > - Paperclip lets people manage AI agents and inspect their saved work. > - Chat feature evaluations test a real request against saved tasks, audit records, links and reload behavior. > - The current suite has Codex and OpenCode cells but no dedicated runnable DeepSeek 1017 backlog plan in TestOps. > - TestOps must select the real case before the workflow can access a model key. > - This PR adds one fixed, default-branch AWS route for native OpenCode and DeepSeek 1017. > - The route keeps original evidence private and sends only a verified result and approved screenshots to TestOps. ## Linked Issues or Issue Description **Subsystem affected** Product E2E evaluation definitions and GitHub Actions delivery. **Problem or motivation** A dedicated TestOps plan cannot run the existing backlog chat journey with native OpenCode. A result must prove saved work and cleanup rather than report that a model call completed. **Proposed solution** Add one OpenCode backlog cell and one fixed public workflow. TestOps selection validation runs in a token-isolated job before builds or inference. The workflow retains private originals and reports the bound outcome to the selected TestOps case. **Alternatives considered** A private-repository wrapper would need a new cross-repository GitHub write credential. The new route uses the existing TestOps GitHub integration instead. Existing manual candidate approval paths remain separate. **Roadmap alignment** This extends the existing feature evaluation infrastructure. It changes no production feature, global runtime default or model default. Current main #18319 supplies the v2 grader, delegation queue accounting and final screenshot capture; these improvements are preserved. Related product work: #16814 and #17126 concern backlog handling. This PR changes neither product implementation and does not claim those changes are required. ## What Changed - Add exactly one native OpenCode backlog cell with DeepSeek 1017 under the separate `runner-opencode-deepseek-1017` profile. Preserve all four existing Codex and production OpenCode cells and their models. - Add a default-branch-only workflow with fixed evaluation, zero-result registration and one plan-launch modes. Require GitHub actor checks and authenticated singleton selection before inference. - Add a separate shipped-main evidence authority. Reuse the existing strict private storage, scanners, inventories and scoped AWS sessions. - Bind sanitized results, generated JUnit and producer-approved screenshots to the exact source, run, attempt and case. Keep failed grades and cleanup failures failed. - Add TestOps API, CLI and result adapters with pure controls and normal unit-test wiring. Keep the reporting token out of the evaluated process. - Verify the current server case result and attachment metadata after upload, without masking an earlier failed upload or evaluation. - Apply only the existing reviewed lock correction, verify unchanged tracked dependency inputs and bind the lock receipt across the compiled-output handoff before frozen installation. - Bind the initial administrator launch to one approved workflow run number and live GitHub source/actor metadata before TestOps authentication. Use bounded REST case reconciliation and only the reviewed existing GitHub integration. - Make two existing CI tests observe their intended state deterministically: a fake clock for copy-feedback lifetime and a completed-pass barrier for retry-cancellation batches. Production behavior and the original assertions remain unchanged. The two test-only fixes are intentionally included because their observation boundaries blocked this delivery’s required normal CI. They preserve every original assertion and change no production behavior. The original failed runs remain recorded, and the exact causes of those observations remain unknown. ## Verification - 157 focused profile, catalog, oracle, filesystem admission, export, private-path and workflow controls passed on the current-main composition. The existing stock OpenCode models and the new exact DeepSeek 1017 selection are checked separately. - 44 API, CLI, filesystem and post-upload readback controls passed, including foreign selection, changed hashes, unsafe files, failed outcomes, expired readback and no-resend behavior. - The new exact private session and trace layout controls produced four genuine parent failures and pass with the selected-path extension. The frozen 244-case candidate approval registry remains unchanged. - The lock guard has seven filesystem controls plus seven existing canonical-owner controls. The unrestricted old workflow failed its actual policy calibration; the corrected frozen-install route passed. - Eval-owning TypeScript checks, workflow lint and Git whitespace checks passed. The separate UI test correction was checked with the focused jsdom unit suite. The server test barrier has pure wrapper and real scheduler controls; its DB-backed route was not run locally. No additional local typecheck/build was run for these test-only changes. - Full native builds, provider execution, app/database behavior and TestOps registration are reserved for GitHub Actions/AWS. They have not run locally. - The initial normal CI run failed the new-workflow policy allowlist and a heartbeat scheduling assertion. The policy fix has an actual parent red/current green control and rejects five unsafe route variants. Heartbeat source is unchanged and its failure cause remains unknown. The original run is retained. - The launch-fence actual caller fails against the published parent and passes the five own-run contradiction controls before authentication. - The prior `a5aa359d` normal CI completed successfully. The complete `f9e1f289` run passed all leaf jobs except an existing copy-feedback UI assertion, plus its dependent verification gate. The four owning UI files were unchanged from the earlier passing head and actual tested merge; the exact scheduling cause remains unknown. - The correction changes only that test to actual React `act` and a scoped fake clock. All eight component tests pass; early-hide and never-clear source variants each fail the intended lifetime assertion. Production clipboard/card behavior is unchanged. - The `2f27f3fc` run passed the corrected UI shard and every leaf job except the existing retry-cancellation intermediate-count assertion, plus its dependent verification gate. The seven relevant owners match the earlier passing run and historical source comparison; the exact CI scheduling cause remains unknown. - The test-only barrier observes a persisted completed worker pass before asserting the unchanged 20-item cap and two-company fairness, then releases it for the original 29-item drain and 180-second idle check. Two extracted-wrapper controls and three actual scheduler controls pass. The real DB-backed test awaits AWS qualification. - On `836e80ac`, both repaired test shards passed in AWS. The original attempt retained a separate E2E cleanup HTTP 500 from a PostgreSQL lock timeout and one runner shutdown. No production repair or precise contention/shutdown cause is claimed. - Exactly one targeted retry of E2E shard 3 and its aggregate gate passed in attempt 2; exactly one recovery of the interrupted server shard and its verification gate passed in attempt 3. Other passed jobs were carried forward. The final normal run is successful. No whole-workflow retry or evaluation/model retry was launched. - Both literal required checks (`ci / verify` and `ci / e2e`) are successful on the exact final source `836e80ac`. Greptile is 5/5 and no review threads remain open. Actual TestOps registration and the OpenCode/DeepSeek feature result remain unqualified until their separate executions. ## Risks - Actual zero-result registration, tenant API shapes and public-repository dispatch access must be checked before the first evaluation. Missing or ambiguous metadata stops the route. - The real OpenCode/DeepSeek backlog result and Allure outcome remain unqualified until that run. No passing result is seeded. - One human query, one accounted work run, 120-second turn, 420-second attempt, 300-second settled-state bound and zero retries remain unchanged. Setup usage is outside work accounting and stays unknown. - The reporting token was configured through the separately reviewed protected credential bootstrap. It is unavailable to AWS evaluation jobs. - Metadata reconciliation is capped at ten pages, 1,000 cases and four parallel detail reads. Missing or inconsistent metadata stops before inference. The initial administrator launch counter must be created once after registration; it does not restrict normal TestOps Run actions. - Existing manual workflows, historical evaluation grades and candidate approval records remain unchanged. The original CI failures are preserved; the test corrections do not establish retrospective causes. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning and tool use. The runtime did not expose its context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
a4308b9791 |
test(e2e): add browser project and task context journey (#18353)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People group tasks into projects and use those projects to keep work in context. > - Deployment journeys already test task creation and assignment. > - They do not yet prove that a new project and its task keep the same saved relationship after reload. > - This pull request adds an explicit browser journey for that gap. > - Independent API reads and visible task properties must agree on the exact records and selected settings. > - The journey uses no model runs. Each deployment must pass before a daily plan can include it. ## Linked Issues or Issue Description **What existing behavior does this improve?** The deployment-targeted Product E2E suite can now check project creation and task context through the production dialogs. **Current behavior** The suite covers standalone task creation and assignment. It does not prove that a newly created project contains the task selected in the composer, or that the relation survives reload. **Proposed behavior** Select `project-task-context`. Create a unique project and a self-assigned task with that project selected. Reopen the project, open its task, and reload. Require exact saved company, project, title, prompt, user assignment and standard work mode. Require no agent runs before or after reload. **Reason and benefit** The same journey can qualify different deployments. It catches lost project links and incorrect saved context without a model call or a separate QA-only test implementation. ## What Changed - Add the browser flow and an independent saved-state oracle with positive and negative cases. - Scope project and task navigation to unique links in the main content so sidebar links cannot bypass list coverage. - Bind reopened routes to saved project and task references. Verify the main project heading and visible task content instead of a hidden duplicate sidebar label. Keep browser failures as closed stage codes. - Register the journey, its closed success checks, and its source files in the definition hash. Default model-matrix selections stay unchanged. - Associate qualification-report coverage with project lifecycle, project task context and task creation. Reject retained browser-only passes containing run IDs, provider/model metadata or native/legacy execution. - Add `project` to the existing durable ownership resource contract. A target provider remains responsible for cleanup of its own project fixtures. - Update discovery expectations, the deployment journey guide and the projects feature map. ## Verification - Focused runner tests: 110 passed across the new oracle, discovery, journey contracts and existing report subprocess checks. The first sandboxed attempt blocked the existing report subprocess IPC socket; the same selection passed with local IPC enabled. - Late review regression checks: both report association and unexpected browser-only model work reproduced as failing tests before the fix. The final contract, saved-state oracle and report catalog selection passed all 72 tests. Runner typecheck passed again after these changes. - Browser readback: the old project-name selector failed in Chromium when a hidden sidebar label preceded the production project heading. All six browser regression cases now pass, including missing rows, wrong project/task destinations and a wrong project property. The fixture uses the production heading and task-row overlay conventions. Credential-free Node 24 imports pass. - Current-master integration: 208 focused catalog, project/search, campaign and reporting tests passed. All nine project/search browser regressions passed. Runner typecheck, map checks and Node 24 imports for both journeys passed. The combined catalog, dispatch, source hashes, success checks, browser-only guard and feature associations retain both selections. - The first deployment qualification created both owned records but failed during later readback with a generic browser code. Authentication, setup, collection and cleanup passed. That failed grade is retained. The corrected source subsequently passed a fresh attached QA campaign; the original failure remains preserved. - New oracle alone: 7 tests passed after the final catalog mapping change. - `node scripts/check-maps.mjs` — passed; structural checks only. - Credential-free `--suite product-journeys --list` — passed and lists this browser journey. - `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json` — passed after building required plugin, runner and eval-kernel declarations. The initial fresh-checkout check failed because those declarations were not built. - All current-head CI gates passed, including repository typecheck, tests, build and browser checks. Greptile reviewed the exact head at 5/5 with no unresolved findings. Fresh attached QA passed at `f19557c9eebde14c2def37195d6b154c23eb9690`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41119722363-1/). All five independent checks and authentication, setup, journey, collection and cleanup passed; zero agent runs started. ## Risks - This adds an opt-in test and changes no production behavior or schema. UI control changes can fail the journey. - The target provider must support durable `project` ownership before it can run this case. Project cleanup must settle owned tasks, reject foreign linked tasks and archive the project while preserving history. - Source registration alone does not include the candidate in any saved daily plan. A fresh pass against that deployment is required first. ## Model Used - OpenAI Codex, GPT-6 family, with code inspection and tool execution. The host does not expose the exact underlying model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no duplicate journey found) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
3b84c89885 |
fix(ci): use complete merge diffs and allow audited documentation (#18362)
## Thinking Path > - Paperclip manages AI agents and needs reliable checks for each contribution. > - CI selects affected suites and then affected Vitest files before allocating runners. > - Recent runs compared a tested merge against a stale event base. > - That included unrelated master changes and sometimes reached the compare API file cap. > - Ordinary documentation also forced every suite and invalidated test caches. > - This PR reads complete merge trees and exempts a small set of audited prose inputs. > - Tests retain conservative coverage when the diff or an input is uncertain. ## Linked Issues or Issue Description Refs #18348 and #18214. This is a follow-up to the merged affected-test selector. Related CI performance work: #16513 and #16500 address setup and suite execution. This PR changes diff collection and input classification. ## What Changed - Fetch the exact tested merge and its parents into a temporary bare repository. Read the complete tree diff against the actual first parent. Validate the second parent against the event PR head. Never execute PR code in the routing job. - Carry the immutable base, merge and PR-head identities into the plan. Validate the same complete diff again before file selection. - Use one prose policy across the planner, Vitest selector and task cache adapter. Exempt only `doc/DEVELOPING.md`, `doc/DATABASE.md`, and Markdown under `doc/plans/`. - Keep the uncached typecheck and CLI guidance guard for the first two documents. Keep policy checks for historical plans. Keep existing coverage for all other docs, runtime skills, fixtures and unknown inputs. - Extend candidate-workflow qualification to exercise mixed source and prose changes. ## Verification - Passed: 15 planner and Git routing tests, including a 353-file diff, stale base, rename, deletion, wrong parent and fetch-failure cases. - Passed: 83 routing, Vitest graph, exact selection, workflow and real Turbo cache checks. - Passed: `pnpm -r typecheck` and `pnpm build`. - Full local `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL startup failures in native-session and Muse tests. It did not pass or complete. The exact cause remains unverified. The changed files do not include those tests or their database setup. Linux CI results are recorded below. - Passed: real mixed Inbox/source/prose graph qualification selected 699/911 workspace test files, preserving the source-only selection. - Passed: both GitHub CI qualification jobs, including real mixed source/prose Vitest selection. Greptile reviewed commit `10ac6f5ec9` at 5/5 with no findings. - First full CI attempt: the [chat shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253342684) received a runner shutdown signal. The [Runner shard](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124253341271) failed the unchanged Antigravity authentication success assertion. All 11 cases in that file passed when run directly. That Runner lane does not use the changed selector/cache path. The exact assertion-failure cause remains unverified. The chat shard passed on the one retry. The authentication assertion also passed on retry, but the same Runner shard failed a different unchanged test: `native-distribution-integrity.test.ts:242`, "retains spawned native bytes until the exact child exits", exceeded 5000ms. The [retry log](https://github.com/paperclipai/paperclip/actions/runs/41111003419/job/124259174201) preserves the failure. The Runner shard and required verify aggregate are still red; all other checks passed. All 33 native-distribution tests then passed in a direct local run. This shows recovery in isolation, not the cause of the Linux timeout. No further blind reruns were issued. - Replayed 15 real recent PR merge commits through a blob-filtered bare fetch. Nine needed a corrected base. The four API comparisons that returned 300 filenames became complete diffs of 3, 57, 66 and 54 files. The 3-file case (#18326) becomes eligible for graph selection. Most other samples still correctly use full CI because they include inputs outside this narrow prose policy. - Cache proof: prose edits preserve a product-test cache hit; changing runtime API documentation invalidates it. ## Risks - Prose exceptions must be removed or revised if product code starts reading those files. The allowlist is deliberately small. Runtime docs and skills remain covered. - A network or Git failure keeps the full-suite fallback. Unexpected merge topology also keeps full CI. Stack and force-full policies remain in place. - This does not claim an end-to-end speedup yet. Many mixed-content PRs still require full CI. The routing fetch reads only shallow commit/tree objects; it does not check out or execute PR files. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and code execution. The exact served model ID and context-window size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1030.0-canary.17 |
||
|
DottaandPaperclip
|
a0e3ac62a7 |
test(evals): add reusable task search and filter journey (#18355)
## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The task list lets people find saved work and open its context. > - Product journeys can exercise the same task UI on local and Cloud targets. > - The shared suite lacks a search and status-filter journey with control tasks. > - This pull request adds that journey with exact task identity checks. > - The benefit is reusable evidence that search, filters, and task navigation work together. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for task search, status filters, and saved task navigation. **Current behavior** The shared journey suite covers task creation and assignment. It does not prove that a search excludes an unrelated task, that a status filter excludes a matching completed task, or that the selected task retains its saved content. **Proposed behavior** An explicit `task-search-filter` selection creates three owned tasks. It searches through the production task list, applies Backlog, and opens the exact target. It verifies reload and Back navigation, saved task fields, and zero agent runs for all three tasks. **Reason and benefit** The same journey can run against a managed local stack or an attached Cloud target. Independent assertions reject missing evidence and plausible wrong results. Deployment qualification and daily-plan admission remain separate gates. **Breaking changes** None. Existing journey selections keep their behavior. Related work: #16979 fixes status filtering before pagination. #15403 covers saved task views. This journey tests the existing search/filter interaction and does not add a new UI or saved-view feature. ## What Changed - Add a zero-run browser journey with one target and two owned controls. - Compare exact visible task identities after search, filtering, reload, and Back navigation. - Verify saved company, owner, title, description, status, and zero agent runs through the public API. - Register dispatch, source hashing, closed success checks, and browser reporting validation. - Add adversarial oracle tests and a campaign dispatch test. Document the selection and its feature-map scope. - Exercise the real browser flow with a stateful UI/API port across Coordinator, combined Tasks, and legacy shells. Inject missing ownership, wrong destinations, lost filters, changed saved context, and unexpected native work. - Use the combined Tasks shell's named Backlog view for reload persistence, as required by its existing view-preset contract. Preserve the successful polled result snapshot and link the case to feature coverage. ## Verification - First attached Cloud attempt failed during Backlog selection. Its cleanup passed and it started no model work. The original failed result is preserved; the corrected candidate passed a fresh attached campaign below. - Local Chromium reproduces the cause: the legacy StatusIcon and text label produce the accessible checkbox name `BacklogBacklog`, so the prior exact `Backlog` lookup finds no status control. - The selector now stays inside the Status group and accepts the explicit and duplicated names. Three real Chromium regressions pass, including a separate assignee called Backlog. - Latest focused journey checks: 130 passed. Runner typecheck and map checks passed. Unexpected browser/API failures retain only closed source-owned stage codes. Screenshot privacy guards and all exact identity, content, filter, and zero-run checks are unchanged. - Fresh attached Cloud qualification passed at `e3dfc0ae37e8ac74e9a07ca24421ea6c96f9bcfc`: [sanitized report](https://pages.paperclip.ing/evals-qa-smoke-41114233380-1/). All five independent search/filter/reload/reopen/no-agent checks and all five phases, including cleanup, passed. The browser journey started zero agent runs. - Runner typecheck passed: `node node_modules/typescript/bin/tsc -p tests/runner-e2e/tsconfig.json`. - Focused oracle, contract, and campaign tests passed: 101 unique tests. The campaign suite was rerun after its new dispatch test: 30 passed. - Direct browser-flow tests passed: 23 tests. These call the real `taskSearchFilter` function through a stateful UI/API port. They do not establish live browser or Cloud qualification. - Latest focused search, oracle, catalog, and report checks passed: 127 tests across five files. - Full runner Vitest completed with 5,204 passes, two skips, and three failures outside this change. Two local embedded PostgreSQL fixtures failed at bootstrap. The Pi controls test expects profile version 35 while the unchanged base declares 37. These failures remain recorded; the PR does not change those fixtures or pins. - The remaining Node support suite passed separately: 137 tests. The standard unit launcher stops before these tests after a Vitest failure. - Credential-free discovery passed: `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/launch.ts --suite product-journeys --list`. - `node scripts/check-maps.mjs` passed. `git diff --check` passed. - All 58 current-head check runs are green, the additional security status passed, Greptile reviewed this exact head at 5/5, and no review threads remain unresolved. Typecheck, test, build, browser CI, and fresh attached Cloud acceptance passed. Daily-plan admission follows merge. ## Risks - Task-list control labels and link structure can change. Such drift fails the journey and needs an update. - The journey creates three board-assigned tasks. It records ownership before creation and records each returned ID immediately. The existing target provider owns cleanup. - Local assertion tests do not prove that a deployed Cloud target passes. The target owner must record a fresh live pass before admission to a daily plan. ## Model Used OpenAI Codex, GPT-6 family. The host does not expose an exact model version or context limit. Used repository search, TypeScript tools, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
8f822bfcf7 |
fix(cloud): persist configuration and load release-image packages (#18335)
Land the approved Cloud configuration and release-image package stack.
Includes the separately reviewed adapter settings (#18335), external adapters (#18333), bundled plugins (#18336), startup configuration (#18339), and migration/package checks (#18341).
Includes the final image-owned loader and linked-dependency inventory corrections.
Verified stack head:
|
||
|
DottaandPaperclip
|
36f4422389 |
build(runner): reduce development debug information (#18307)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents and contributors compile the Rust runner inside workspaces with limited storage. > - Default development builds include full debug information in many test executables. > - A controlled Linux build used 11.54 GiB with those defaults. > - This pull request uses line-only debug information for the development profile and its test profile. > - The measured build used 4.35 GiB and retained source-line information for backtraces. ## Linked Issues or Issue Description **What existing behavior does this improve?** The storage used by Rust development builds and default-profile test builds in `packages/paperclip-runner`. **Current behavior** The workspace uses Cargo's full development debug information. A clean Linux ARM64 comparison on source `d2280108f56872a6052a5011649a73cef9aac206`, Rust 1.97.1, and four build jobs used 11.54 GiB for the test build output. This can exceed a small sandbox's entire disk allocation. **Proposed behavior** Set `[profile.dev] debug = "line-tables-only"` in the Rust workspace manifest. The default test profile inherits it. The same comparison used 4.35 GiB, a 62.3% reduction. Incremental compilation stays enabled. Release settings stay as they are. **Reason and benefit** Reduce build storage while preserving file names and line numbers in backtraces. This avoids removing all debug information or changing which tests compile. **Breaking changes** Default development builds omit type and local-variable debugger information. Contributors can restore it with `CARGO_PROFILE_DEV_DEBUG=2` and `CARGO_PROFILE_TEST_DEBUG=2`. There are no product API changes. Related: #18172 handles workspace export after disk exhaustion. #16500 proposes separate CI-only debug settings and test scheduling changes. This PR sets the workspace default and remains compatible with explicit environment overrides. Searches found no existing PR for `line-tables-only`. ## What Changed - Set the runner workspace development debug level to `line-tables-only`. - Document test-profile inheritance and the full-debug environment overrides. ## Verification - Prior controlled comparison: fresh target directory per setting, shared dependency downloads outside timing, identical source and lockfile, Rust/Cargo 1.97.1, Linux ARM64, four build jobs, and a six-GiB memory limit. First compile library tests, then all test targets with `cargo test --locked --offline -p paperclip-runner-core --no-run durable::`. Both configurations compiled all 32 test executables. Measure the target directory with `du -sk`, counting hard links once. Full debug: 11.54 GiB. Line-only debug: 4.35 GiB. These numbers exclude source, registry, image, and Node dependencies. The initial benchmark setup omitted an external fixture; that run was discarded after adding the fixture from the same source revision. - Current manifest verification passed on Linux ARM64 / Rust 1.97.1 with profile environment overrides unset: all 32 test executables compile, all 98 durable unit tests pass, and development binaries build. Verbose compiler commands show `-C debuginfo=line-tables-only` and an incremental directory for both the library and its test harness. This verifies test-profile inheritance from the manifest. The fresh target directory is 4.36 GiB on the updated source. No new test asserts the manifest text; validation uses the actual compiler and existing tests. - `cargo metadata --manifest-path packages/paperclip-runner/runner/Cargo.toml --locked --offline --no-deps --format-version 1` and `git diff --check` pass. - [CI passed on commit `14d798139f5ff04414c6acbe15cec04e9b680a00`](https://github.com/paperclipai/paperclip/actions/runs/41059532543), including the `ci / verify` aggregate. The separate native Docker Runner check also passed. The final check inventory has 55 successful checks, four expected skips, and the neutral review described below; the legacy security status passed. Two first-attempt worker interruptions required one targeted retry each: Runner Vitest 2/2 received a shutdown signal during native integration verification, and chat 1/3 lost communication with GitHub. The exact interruption causes remain unknown. The Runner retry passed 132 files / 2,321 tests (two existing skips), then four native integration files / 127 tests (five existing skips). The chat retry and final aggregate passed. The successful retries establish recovery on this commit; they do not establish the cause of either original interruption. - Greptile reviewed the current commit at 5/5 with no findings. The separate Paperclip review execution did not complete and returned a neutral check without an assessment. This is not counted as a completed review. ## Risks - Native debugger inspection of types and local variables needs a full-debug override. Source-line backtraces remain available. - Changing debug settings rebuilds affected artifacts. Existing full-debug outputs may remain until the contributor cleans their target directory. - The measured savings are from Linux ARM64 and the cited source revision. Other platforms and workloads can differ. This is a build-profile change, not a sandbox capacity increase. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, shell execution, and Linux build measurements. The session does not expose a more specific serving model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandClaude Opus 5.5
|
8362dbff4d |
feat(runner): make every ACPX provider declare its release attestation (#18342)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native ACPX runs admit a provider only when its saved profile version and `commandDigest` match the current profile exactly. > - #18325 made every provider digest attest its upstream release, and bound the shared ACPX layer and Runner source through the pack `contractDigest`. > - The model is documented, but it lived in prose. A new provider, such as one in an open PR, could still choose another digest model without any check or guidance. > - This pull request makes the model the default path: each provider declares its attestation in one registry, and the generator, a test and a scoped rule explain the pattern where contributors meet it. > - The benefit is that new providers and upstream bumps follow one model, without a growing list of blocked legacy patterns. ## Linked Issues or Issue Description Refs #18325, #18237. **What happened?** After #18325, only Copilot, Grok and Pi went through `acpx-release-attestations.mjs`. Cursor and Antigravity had separate inline checks in the generator, and Claude and Codex had none. A new provider could add a profile with any digest. Nothing told the author how to compute it, and nothing pinned the profile versions that now change only for deliberate resets. **Expected behavior** Every ACPX provider declares how its `commandDigest` attests the upstream release. Tools and review guidance describe the pattern when a contributor adds a provider, bumps an upstream release or changes a version. **Steps to reproduce** 1. Add a new provider profile to `packages/paperclip-runner/acpx-profiles.json` with an arbitrary `commandDigest`. 2. Run `node packages/paperclip-runner/scripts/generate-acpx-profiles.mjs --check`. 3. Before this change, the check passes. ## What Changed - Add `ACPX_RELEASE_ATTESTATIONS` to `scripts/acpx-release-attestations.mjs`. Every profile declares one kind: - `installed-command` (Claude, Codex): installation hashes the upstream launch command. - `launcher` (Grok): the SHA-256 of the launcher shipped verbatim. - `closure` (Copilot, Pi): upstream versions and complete platform closure pins. - `declaration` (Cursor, Antigravity): a checked-in release declaration, as canonical JSON. - `generate-acpx-profiles.mjs` checks every profile through the registry. The separate Cursor and Antigravity digest checks are now part of the same path. No digest changes. - A profile without a declared attestation fails generation. The message says to declare one and points to the rule. - A stale digest message now says that a digest changes only with its upstream release, and that an upstream change keeps the profile version. - Add a test that pins the profile versions. It explains that a bump resets every saved session for that provider, and that a new provider starts at 1. - Add `rules/acpx-profile-identity.mdc`. It describes the identity model, how to add a provider, when to bump a version, and review criteria. `AGENTS.md` and `.greptile/files.json` route the profile, attestation, contract-registry, ACPX driver and provider paths to it. - Update the profile identity section in `packages/paperclip-runner/docs/durable-recovery.md` with the attestation kinds and the version policy. ## Verification - In `packages/paperclip-runner`: - `node scripts/generate-acpx-profiles.mjs --check` passes. Running it without `--check` changes no digest. - `node --test scripts/acpx-release-attestations.test.mjs scripts/contracts/contract-fingerprints.test.mjs`: 22 tests pass. New cases: every profile declares an attestation, a new provider without one fails with guidance, and Cursor and Antigravity declaration changes affect only their own digests. - `vitest run src/drivers/acpx/profile-compatibility.test.ts`: 15 tests pass, including the version table. - `tsc -p tsconfig.json --noEmit` passes. ## Risks - Low risk. No digest, version or runtime admission behavior changes. - A new provider PR must add one registry entry. The generator error states what to add. - The version table needs an edit for each deliberate reset. Upstream bumps do not touch it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>canary/v2026.1030.0-canary.16 |
||
|
github-actions[bot]andlockfile-bot
|
ade402c75f |
chore(lockfile): refresh pnpm-lock.yaml (#18360)
## Thinking Path > - Paperclip uses a dependency lock for reproducible builds. > - Dependency changes on master need matching lock entries. > - Paid eval reports also pin a reviewed offline correction. > - Reports must install before paid model work starts. > - This automated PR keeps the lock and report proof together. ## Linked Issues or Issue Description **What happened?** Dependency manifests on master changed after the last committed lock refresh. **Expected behavior** Frozen installs must match those manifests. Eval reports must retain matching offline lock proof. **Steps to reproduce** Run a frozen install before and after this lock refresh. ## What Changed - Refresh the dependency lock from the manifests on master. - Include the matching offline report correction and hashes. ## Verification - Pinned pnpm resolution completed with lifecycle scripts disabled. - The report proof refresh accepted the reviewed committed baseline. - The workflow rejected changes outside the lock and report proof files. ## Risks The lockfile bot merges this PR without CI or review when every added package version is at least 3 days old on npm. Otherwise the PR waits for a maintainer, and the daily refresh tries again. The bot merges only the exact commit this workflow generated from manifests already reviewed on master. Unknown report lock baselines remain blocked. This update changes no permissions or paid enable gates. ## Model Used None — generated by pnpm and the deterministic report-lock updater. ## Checklist - [x] Thinking path and problem description included - [x] Generated lock and matching report proof included - [x] Release-age gate decides between bot merge and maintainer review Co-authored-by: lockfile-bot <lockfile-bot@users.noreply.github.com> |
||
|
DottaandPaperclip
|
52e846f2f3 |
perf(ci): select affected Vitest files before allocating test runners (#18348)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its CI must catch regressions without running unrelated work on every pull request. > - The existing task planner selects package lanes, but each selected lane still runs all its test files. > - Vitest can find related imports, but that graph misses tests that read files or launch processes. > - This pull request combines the real Vitest graph with conservative dependency checks before CI allocates test runners. > - The result runs fewer test files and can allocate fewer jobs while retaining tests with unknown dependencies. ## Linked Issues or Issue Description Refs #18214. The repository and PR search found no duplicate file-selection implementation. **What existing behavior does this improve?** The general and serialized Vitest lanes in pull request CI. **Current behavior** An affected package schedules its complete test lane and its fixed shard count. **Proposed behavior** For source-only diffs, select related tests plus conservative filesystem, process, network and opaque consumers. Repartition those files within their existing lane owners. Keep the complete original plan when selection cannot be trusted. **Reason and benefit** Avoid unrelated test execution and allocate fewer runners when the selected workload fits fewer existing job budgets. Report file counts and job counts separately. **Breaking changes** No product or API changes. The set of PR test files and jobs changes for eligible diffs. Full CI and the kill switch retain the original coverage. ## What Changed - Add a separate selection job on the gate-approved test fleet. The metadata gate does not load PR code. - Inspect real Vitest/Vite dependency edges. Retain opaque consumers and unchanged downstream projects. Respect only complete local mock factories. Reject unknown setup dependencies and source mutations during dependency setup. - Declare `oxc-parser` directly for AST inspection. This exact version is already a transitive dependency in the checked-in lockfile; the direct declaration avoids relying on pnpm's internal dependency layout. - Produce a merge-SHA-bound manifest with exact lane ownership and duration-aware partitions. Preserve serialized execution, chat shards and native Runner ownership. - Execute exact Vitest test specifications. Positional substring filters cannot accidentally run a similarly named file from another shard. - Use the original matrix if selection fails or times out. Fail selected jobs if their published artifact is missing or invalid. - Save the uploaded artifact ID in the executable plan. Failed-job retries reuse that artifact even when GitHub increments the attempt number. - Hash selected assignments into Turbo cache inputs. A subset cannot satisfy a full run or another subset. - Add graph, workflow, real-runner and real-Turbo qualification. The regression fixture proves a filesystem contract fails even when raw Vitest selection misses it. - Document selection limits, audit artifacts and `CI_VITEST_SELECTION=off`. ## Verification - All 138 affected routing, sharding, workflow, manifest, real Vitest and real Turbo checks passed under Node 24 on the final implementation. Additional release workflow checks passed. Cases cover opaque setup, remocking, generated inputs, Node's builtin loader, artifact retries, and exact execution of similarly named files. - Repository-wide `pnpm -r typecheck` and `pnpm build` passed before the clean rebase onto current master. The full remote typecheck and build are required on the current head. - `actionlint` passed for both changed workflows. `git diff --check` passed. - The Linux qualification run passed and selected 699 of 911 workspace test files for an Inbox change. UI graph construction took 16.1 seconds. This still uses two jobs. Synthetic partition coverage proves a 15-of-120 selection uses two of twelve jobs without dropping or duplicating files. - Both selected workspace shards passed locally. After tightening execution to exact specifications, the real colliding UI test ran as exactly one file with two passing assertions; the real server auth middleware test ran as exactly one file with 28 passing assertions. - The server probe selected 1,158 of 1,260 general files and 161 of 163 serialized files for an auth middleware change. Those conservative selections retain 12 general and nine serialized jobs. The server's temporary-home setup exception is pinned to reviewed content hashes; changed setup fails closed. - Full `pnpm test:run` has not been run locally. The full PR test matrix is required because this change modifies global CI inputs. Final-head CI is pending at runner allocation, and both qualification jobs are queued. Other recent repository workflows are also queued. Superseded runs of this PR were cancelled; this PR remains in draft until the required checks execute. - Greptile reviewed final commit `31b484fc5b412da245cffb52a8ac8977228c5651` at 5/5 with no unresolved findings. The earlier qualification run passed; it does not replace checks on the final commit. ## Risks - Dependency analysis adds setup and graph time. Fewer files do not guarantee a shorter critical path. CI summaries expose the actual file and job counts. - Static dependency graphs cannot describe all runtime inputs. Unknown inputs retain tests or use the original plan. The pure-package allowlist requires conservative review. - The trusted workflow runs from master. This PR exercises the new selector in the qualification workflow; ordinary PR selection starts after merge. Set `CI_VITEST_SELECTION=off`, use `full-ci`, or select full/shadow mode to disable narrowing. - The lockfile remains managed by the existing repository actions and is not changed here. ## Model Used OpenAI Codex, based on GPT-6, with repository tools and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
aa58fa1073 |
perf(ui): restore fast Coordinator task and inbox loading (#18323)
## Thinking Path > - Paperclip lets people manage work done by agents. > - Tasks, Inbox, and Needs You must stay usable as work history grows. > - The Coordinator views bypassed some existing collection optimizations. > - Navigation fetched complete decision cards for a count, and Tasks mounted every loaded row. > - This PR uses the canonical attention rules for small count and task-ID responses. It also restores bounded rendering and the optimized Inbox selection. > - The benefit is less data and less initial rendering work without a second definition of pending work. ## Linked Issues or Issue Description **What happened?** Coordinator navigation fetched the full attention feed on every cold collection visit. Tasks rendered the entire loaded collection. Coordinator could also select the old Inbox loader when Streamlined UI was off. **Expected behavior** Fetch only the projection needed by each surface. Start with a bounded set of rows. Keep complete task membership, accurate counts, current permissions, and stable loaded rows. **Steps to reproduce** 1. Enable Coordinator and open Tasks or Inbox with an empty browser query cache. 2. Observe a complete attention-card download for the navigation badge. 3. Open a large task list and observe that every loaded row mounts at once. 4. Disable Streamlined UI while keeping Coordinator enabled, then open Mine and observe three overlapping compact task reads. Related work: #17756, #3744, #17983, and #18312. ## What Changed - Add board-only attention count and complete task-ID projections. Reuse canonical company, privacy, addressee, deduplication, dismissal, snooze, and archive rules. Skip card bodies, plan/image reads, and presentation roster lookup. - Share the small count read across desktop and mobile navigation. Reuse the page-owned response on Needs You and Needs attention. Keep the newest known count through navigation, including zero. - Use the task-ID index for Needs attention and keep task row hydration in batches of at most 100. - Render 100 task rows initially, across the whole expanded tree. Reveal 150 more at a time through scrolling or a button. Keep existing rows through refresh and append. Defer offscreen leaf layout with shared tokens. - Keep Coordinator on the optimized Inbox when Streamlined UI is off. Use index reads for expanded project recents. - Add real-database, component, and production-build browser regression checks. Update the access and feature maps. ## Verification - All required CI checks pass on `1c68ac8957490a6c8585ba0939355f536c8120bf`: typecheck, build, server/workspace tests, serialized database suites, and eight E2E shards. Greptile is **5/5**, with its row-retention finding fixed and the review thread resolved. Storybook checks were intentionally skipped; Paperclip Review completed neutral. - Workspace typecheck and production build passed locally. The final rebased source also passed UI typecheck and production build. Token gates and map checks passed. - The attention service suite passed all 34 tests on disposable PostgreSQL. Final targeted checks cover compact projections, complete-set counts, quality approvals, snoozes, private tasks, and board/company access. Database response capture confirms compact projections do not read plans or images. All 31 OpenAPI tests passed after registering both endpoint contracts. - Six production-build browser journeys passed: desktop, mobile, Coordinator Inbox with Streamlined UI off, Needs You, Needs attention, and an older server page followed by task open/Back. They verify the initial 100-row budget, scrolling, retained DOM identity/height, one owner per attention read, complete task membership, 220 retained rows on Back, and less than 2 px of return-position difference. - The row-budget and Inbox regressions fail against original source and pass with this change. The final three affected UI suites passed 40 tests, and the broader Inbox/IssuesList suites passed 101 tests serially. ### Controlled production-build comparison Baseline `b31ec0a2dc079e32272a8f618425fe92b68bbaec` versus PR head `1c68ac8957490a6c8585ba0939355f536c8120bf`. Isolated baseline asset `index-CS9uPxMx.js`; candidate asset `index-BxJeTQ30.js`. Both UI builds used the same warm disposable API server and database, with 220 tasks and 110 pending board approvals. Chromium **151.0.7922.34**, 1440 × 900 viewport, **4× CPU throttle**, loopback with no added network latency, alternating baseline/candidate order. Cold cases use a new browser context with network cache disabled; warm cases navigate within the same context. Other host workloads are uncontrolled. Readiness requires the destination collection's first visible task row, then two animation frames. Inbox readiness waits for its canonical Mine route so previous-page rows cannot satisfy the measurement. Five paired samples per case; cold Inbox has a second five-pair confirmation wave because the first result was slower. Times below are **milliseconds**; parentheses show min–max. Paired savings are the median of individual before-minus-after pairs, not the difference between medians. | Journey | Pairs | Baseline median (range) | Candidate median (range) | Median paired saving | Faster pairs | | --- | ---: | ---: | ---: | ---: | ---: | | Cold Tasks | 5 | 2752 (2542–3025) | 2300 (2275–2524) | 328 | 5/5 | | Cold Inbox | 10 | 2536 (2283–2927) | 2582 (2117–2856) | 37 | 5/10 | | Warm Tasks navigation | 5 | 2054 (1826–2479) | 1448 (1419–1548) | 635 | 5/5 | | Warm Inbox navigation | 5 | 2020 (1485–2254) | 1568 (1289–1848) | 350 | 5/5 | | Task → Back to Tasks | 5 | 1747 (1609–1948) | 1327 (1238–1355) | 472 | 5/5 | Cold Tasks, warm Tasks, warm Inbox, and Back improved in every measured pair. **Cold Inbox latency is inconclusive:** the two waves pointed in opposite directions, with the candidate faster in 5/10 pairs. This PR claims less badge work there, not a proven cold Inbox latency improvement. The canonical badge response shrank from **200,039 to 359 decoded bytes** (encoded HTTP transfer: **8,125 to 598 bytes**). The compact response also skips card payloads, plans, images, and presentation-only roster work. These are measured mechanism reductions, separate from end-to-end latency. Needs You still owns its full card feed; Needs attention owns its complete task-ID projection. Reproduction setup and assertions are in `tests/perf/collections/README.md`. <details> <summary>Individual paired readiness measurements (ms)</summary> | Journey | Pair | Baseline | Candidate | | --- | ---: | ---: | ---: | | Cold Tasks | 1 | 3025 | 2283 | | Cold Tasks | 2 | 2542 | 2275 | | Cold Tasks | 3 | 2752 | 2424 | | Cold Tasks | 4 | 2783 | 2300 | | Cold Tasks | 5 | 2641 | 2524 | | Cold Inbox | 1 | 2927 | 2702 | | Cold Inbox | 2 | 2422 | 2629 | | Cold Inbox | 3 | 2409 | 2606 | | Cold Inbox | 4 | 2621 | 2769 | | Cold Inbox | 5 | 2539 | 2856 | | Cold Inbox | 6 | 2283 | 2117 | | Cold Inbox | 7 | 2466 | 2558 | | Cold Inbox | 8 | 2674 | 2286 | | Cold Inbox | 9 | 2696 | 2389 | | Cold Inbox | 10 | 2533 | 2183 | | Warm Tasks navigation | 1 | 2479 | 1423 | | Warm Tasks navigation | 2 | 2228 | 1548 | | Warm Tasks navigation | 3 | 2054 | 1419 | | Warm Tasks navigation | 4 | 1826 | 1448 | | Warm Tasks navigation | 5 | 1954 | 1518 | | Warm Inbox navigation | 1 | 2020 | 1661 | | Warm Inbox navigation | 2 | 2198 | 1848 | | Warm Inbox navigation | 3 | 1733 | 1497 | | Warm Inbox navigation | 4 | 2254 | 1568 | | Warm Inbox navigation | 5 | 1485 | 1289 | | Task → Back to Tasks | 1 | 1721 | 1330 | | Task → Back to Tasks | 2 | 1747 | 1275 | | Task → Back to Tasks | 3 | 1948 | 1355 | | Task → Back to Tasks | 4 | 1609 | 1327 | | Task → Back to Tasks | 5 | 1790 | 1238 | </details> ### Failed attempts and limits - The local full repository command hit embedded PostgreSQL startup failures and 16 failures in the untouched native session executor suite; it was stopped. The full local suite is not claimed green. CI on the final revision provides the broad test result. - A concurrent local UI run hit timing failures under host contention. Inbox/IssuesList subsequently passed all 101 tests serially. No tests were weakened to address these timeouts. - CI caught two integration gaps introduced during this work: missing OpenAPI entries for the new endpoints, and an attention fixture incompatible with the routine filters added on master. Both were fixed; the affected 31 OpenAPI and 40 UI tests pass. - The final revision's first CI attempt failed two untouched tests: scheduled-retry batch cancellation and the tool-access audience/archive race. Both failed jobs passed when rerun without changing the revision. Their original failed attempts remain visible in [the workflow history](https://github.com/paperclipai/paperclip/actions/runs/41083379875). - Browser evidence uses synthetic disposable data. This PR is not deployed staging or production verification. ## Risks - New attention sources must preserve canonical eligibility. Both projections call the existing collector, so they share its permissions and complete-set behavior. - The count and task-ID caches must not stand in for full decision cards. Each projection has its own key under the existing attention invalidation prefix. - A row budget can omit descendants if applied per parent. The tests cover one global budget across a large expanded tree, refresh, append, reset, and incremental loading. - Local browser evidence uses synthetic data and is not staging deployment proof. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend model deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites; broad-run limitations are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1030.0-canary.15 |
||
|
DottaandPaperclip
|
ebd05e9f4c |
perf: avoid scanning unused provider pack trees during archive upload (#18340)
## Thinking Path > - Paperclip manages AI agents and their work. > - Native agents need verified provider files in remote sandboxes. > - A fresh sandbox can need the bundled provider-pack archive. > - The upload path also hashes the expanded controller tree, which it does not upload. > - That synchronous scan blocks the server event loop and delays other requests. > - This change verifies the archive source and the installed pack, and keeps source-tree checks for directory uploads. ## Linked Issues or Issue Description **What happened?** Provider-pack archive fallback scans and hashes the full expanded controller tree before reading the build archive. The archive is the source sent to the sandbox. The extra scan blocks the Node event loop for several seconds. An overlapping asynchronous operation can then appear to take as long as that scan. **Expected behavior** Verify the bytes used by the selected upload path. Keep full verification of the installed pack against the selected manifest before a provider can use it. **Steps to reproduce** 1. Use a packaged provider pack with a build archive. 2. Start a remote native run whose sandbox needs that pack. 3. Inspect `provider_pack.archive.verify` and event-loop delay on the controller. Related: #18308 adds timing detail for workspace preparation. A search found no duplicate archive-verification fix. ## What Changed - Use the verified archive without scanning the unused expanded directory. - Keep full directory verification when no archive exists or the remote lacks archive support. - Keep archive digest checks and full verification of installed bytes. - Add `provider_pack.directory.verify` for directory fallback. - Add regression cases for both archive formats, fallback, and final verification failure. ## Verification - Both archive regressions failed before the change and passed after it. - 80 focused archive, selection, and verification cases pass on the rebased branch. - 20 native manifest and admission cases pass. - Server typecheck, server build, and the runtime bundle pass. - A disposable sandbox from the recorded image exercised the real uploader and full installed-pack verifier. It accepted the unchanged build bytes and rejected an altered proxy file. No model was called. The sandbox was deleted and its absence was verified. - A local baseline/archive/archive/baseline probe used the same unchanged public build pack. Preparation took 30.673 s, 84 ms, 106 ms, and 5.371 s. The slow baseline outlier is retained. This measures one controller phase; it does not establish a whole-turn or hosted speedup. ## Risks The archive path no longer checks expanded files that it does not send. Archive checks still fail closed, and the installed files must pass the same full manifest verification. Directory fallback keeps its source-tree check. Installs without archives retain that synchronous scan. The local sandbox probe tests upload and installed-content checks; it does not qualify admission of an older pack by a newer controller. ## Model Used OpenAI Codex, GPT-6 family, with tool use and local code execution. The exact runtime model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
6d61990692 |
fix: allow managed Git helpers to select a private interpreter (#18337)
## Thinking Path > - Paperclip manages AI agents and their execution environments. > - Managed Git helpers run next to those agents. > - Some environments keep their Node interpreter outside the agent PATH. > - The current generator always emits an env-based Node shebang. > - This pull request lets the environment select an absolute interpreter path. > - Existing callers keep the same default behavior. ## Linked Issues or Issue Description Refs #18212. **What happened?** A managed helper can fail when Node exists only as a private runtime binary. The caller must currently alter generated source or expose that runtime through PATH. **Expected behavior** The source generator accepts an optional interpreter path. The environment can keep its own runtime private. **Steps to reproduce** Generate a helper in an environment whose PATH has no usable Node. Run a local Git command through that helper. The env-based shebang cannot select the private interpreter. ## What Changed - Add optional `interpreterPath` to `githubLauncherSource`. - Preserve `/usr/bin/env node` when the option is absent. - Reject relative paths, whitespace and NUL bytes. - Document the option and test actual execution with an unusable Node on PATH. ## Verification - The affected adapter-utils package typecheck passed. - All 26 tests in `packages/adapter-utils/src/github-launcher.test.ts` passed. - The new test runs a generated helper as a child process. It proves that the selected interpreter bypasses a failing PATH-selected Node and that the helper body stays unchanged. - Invalid-path tests passed. Existing broker and local-Git tests passed. - Full checks run in GitHub CI per the user's request. No full local suite was run. ## Risks The option changes only the shebang. Callers own interpreter installation and immutability. Default behavior, PATH, credentials and permission policy stay unchanged. No schema or workflow changes. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`), high reasoning, with source inspection and targeted execution. The host does not expose an exact context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>canary/v2026.1030.0-canary.14 |
||
|
Nicky LeachandClaude Opus 5.5
|
abc270ee4c |
fix(runner): attest upstream releases and bind ACPX uniformly for every harness (#18325)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native ACPX runs admit a provider only when its saved profile and `commandDigest` match the current profile exactly. > - For Copilot, Grok and Pi, `commandDigest` hashed dozens of Paperclip source files. Claude, Codex, Cursor and Antigravity hash only their upstream release. > - Any change to a bound source therefore advanced these profiles. #18212 alone moved Copilot from v88 to v91 and Grok from v15 to v19. Each bump reset in-flight sessions and added immutable identity fixtures (about 160 on master). > - #18237 added the provider pack `contractDigest`, which now guards Runner source compatibility at the pack boundary. > - This pull request changes the Copilot, Grok and Pi digests to upstream release attestations, as the other providers already use. > - It also binds the shared ACPX layer the same way for every ACPX harness: the profile's `acpxVersion` and the pack `contractDigest`, which now covers the Runner's ACPX patch. > - The benefit is that Runner source changes no longer reset these sessions or need new profile versions and fixtures. ## Linked Issues or Issue Description Refs #18237, #18212, #17972. **What happened?** A Runner change that touched a bound source file changed the Copilot, Grok or Pi `commandDigest`. The author had to advance the profile version, regenerate an identity fixture and keep every old fixture. On deploy, saved sessions for that provider failed the exact admission check and started fresh. **Expected behavior** A provider's `commandDigest` changes only when its upstream release changes: an upstream version, a platform closure, the ACPX patch or the Grok launcher. Runner source compatibility is the job of the pack `contractDigest`. **Steps to reproduce** 1. On master, change a comment in `src/drivers/acpx/runtime-host.ts`. 2. Run `node packages/paperclip-runner/scripts/grok-profile-identity.mjs --check`. 3. The check fails. The Grok profile must advance, and the next deploy resets Grok sessions. ## What Changed - Add `packages/paperclip-runner/scripts/acpx-release-attestations.mjs`. It computes the release attestation for each provider: - Copilot and Pi: a canonical-JSON hash of the profile version, upstream package versions and the platform closure pins (Pi also binds its Node version). - Grok: the SHA-256 of `launcher.cjs`. Admission already hashes these bytes. - The attestation binds no Paperclip source, shared ACPX layer, absolute path, pack manifest digest or install layout. - Bind the shared ACPX layer once for every ACPX harness (Claude, Codex, Copilot, Cursor, Grok, Pi, Antigravity): - Add the Runner's ACPX patch (`patches/acpx@<runner acpx version>.patch`) to the pack `contractDigest`. A patch change now retires older image packs for every harness. Other packages' ACPX patches are not read. A build context without the Runner's patch fails. - Remove the ACPX patch hash from Cursor's release declaration (`cursor-contract.json`) and advance Cursor once to profile 17. Copilot, Pi and Cursor no longer bind ACPX in their own digests, like Claude, Codex, Grok and Antigravity. - Saved inputs for every harness already must match the current `acpxVersion`. - Claude and Codex digests do not change. - `generate-acpx-profiles.mjs` writes these digests to `acpx-profiles.json`. `check:acpx-profiles` refuses a stale digest. - Move the Copilot closure pins to `copilot-closure-pins.ts`, so that the generator can import them. `copilot-installation.ts` re-exports them. - Advance each profile once and then freeze it: Copilot 92, Grok 20, Pi 37. A version change at the switch keeps the rollout on the same path as every earlier profile bump. - Replace the historical version lists in `profile-compatibility.ts` with a range. Every revision from 1 through the current one decodes. Claude and Codex still decode revisions 1 through 5. Launch and recovery admission still require the exact current profile and digest. - Delete: - `scripts/copilot-profile-identity.mjs`, `scripts/grok-profile-identity.mjs` and its test - the Grok launcher source stamp - 159 Copilot, Grok and Pi identity fixtures, including three unreferenced `copilot-mainline-v*` fixtures. The Cursor fixtures stay. - Rewrite the affected tests as behaviour checks: - Each provider admits only the exact current profile. Every earlier version decodes and is refused at admission. - The last source-bound digests, and a changed digest at the current version, are refused by recovery and by the backend factory. - The Rust Pi descriptor test uses the generated profile instead of fixtures. - New `acpx-release-attestations.test.mjs`: a relocated copy of the inputs gives the same digests, each upstream input changes only its own providers, and the generator refreshes and checks the manifest. - Remove the fixture exceptions from `.dockerignore`. Add the attestation script and the generator to the Daytona image content inputs. - Update the Pi install probe and the Rust integration constant for the new Pi digest. - Update the docs to describe the current attestation model: `doc/grok-native-runner.md`, `doc/RUNNER-INTEGRATION-PLAYBOOK.md`, `doc/observability.md`, `packages/paperclip-runner/docs/durable-recovery.md`, and the Copilot, Pi and rich ACP architecture reports. Each report now opens with the current profile identity; earlier source-bound profiles remain as dated chronology. No doc references a deleted file. ## Verification - In `packages/paperclip-runner`: - `node --test scripts/acpx-release-attestations.test.mjs scripts/contracts/contract-fingerprints.test.mjs`: 19 tests pass, including a Runner ACPX patch edit that changes the contract digest, other patch edits (including `acpx@0.12.0` for `adapter-utils`) that do not, and a build context without the patch that fails. - `node scripts/generate-acpx-profiles.mjs --check` passes. - `vitest run src/drivers/acpx src/backends src/contracts`: 1647 tests pass, 9 skipped. - `tsc -p tsconfig.json --noEmit` passes. - From the repository root, `vitest run --config tests/runner-e2e/vitest.config.ts tests/runner-e2e/daytona-image.test.ts`: 11 tests pass. - In `server`, the 8 test files that use ACPX profiles, pack digests or `contractDigest`: 1617 tests pass. - Not run locally: the Rust tests and `cargo fmt`, because this machine has no Rust toolchain. CI runs them. - Before this change, a synthetic upgrade check built two Runner revisions that differ only in shared source. A Claude checkpoint saved under the first revision rebinds under the second, and a changed Claude `commandDigest` starts fresh. This confirms that the provider JSON has no other build-dependent field. ## Risks - **One reset at rollout.** The Copilot, Grok, Pi and Cursor digests and versions change once. Saved sessions for these providers start fresh on their next run, as after any earlier profile bump. After this change, Runner source changes no longer cause this reset. - **Lost "source changed, so requalify" signal.** The old digest forced a new profile version when bound source changed. Nothing enforced requalification: Copilot was last qualified at v47. Pack-boundary contract changes are now caught by the pack `contractDigest`. - **Image packs are retired once for every provider.** Adding the ACPX patch changes the pack `contractDigest`, so the controller uploads its bundled pack (about 16 s per remote start) until images are refreshed. Claude and Codex sessions are not reset: their digests and versions do not change. - **Grok launcher and closure pins.** The Grok digest covers only the launcher bytes. Grok's runtime version is still checked separately at admission, as before. - **Grok digest equals early Grok profiles.** The launcher bytes have not changed since profile 1, so the new Grok digest equals the published profile 1 digest. The profile version still separates them: launch, recovery and the backend factory require profile 20. An image pack from that period has no `contractDigest`, so the pack gate refuses it. `remote-provider-pack-verification.test.ts` now covers this case. - **Docker context checks** run `generate-acpx-profiles.mjs --check`, which now imports `.ts` pin files through Node type stripping. `materialize-pi-distribution.mjs` already relies on this on Node 24. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
f5e5b05a33 |
Preserve native workspace recovery across Stop and sandbox reuse (#18332)
Land the thirteen-file native recovery fix independently from #18245. Retain verified Stop evidence and queued user comments. Retire stale preparation ownership atomically and require paired sandbox lease handoff receipts before workspace recovery. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
75666224d4 |
fix(native): use the deadline for remote health retries (#18324)
## Thinking Path > - Paperclip manages agent work and its execution state. > - Remote Runner processes use health reads to check their saved process identity. > - A failed health request cannot prove that the process stopped. > - Three fast gateway errors can exhaust the old retry limit in half a second. > - The caller already gives these reads a shared 30-second recovery budget. > - This pull request uses that budget with backoff for read-only failures. > - A short gateway outage can then clear without stopping the existing turn. ## Linked Issues or Issue Description **What happened?** A transient HTTP 502 during sandbox process supervision caused an active native Codex turn to fail. The probe stopped after three attempts, even when most of its 30-second budget remained. The run reported `notification_transport_failed` and required workspace recovery. **Expected behavior** Retry read-only transport failures within the existing deadline. Keep process identity conflicts authoritative. Do not repeat a signal after a lost acknowledgement. **Steps to reproduce** 1. Start a read-only process verification with a 30-second deadline. 2. Return HTTP 502 from its first three transport reads, then allow the next read to succeed. 3. The old helper rejects after three reads. The fixed helper returns the successful observation. Related repairs: #16451 introduced bounded remote probes. #16835 and #17799 require proof of process exit before replacement. This change retains those guards. ## What Changed - Use the existing absolute deadline instead of a three-attempt limit for read-only probe failures. - Back off from 250 milliseconds to two seconds. Clip each request to ten seconds or the remaining budget. - Preserve the original error and diagnostic when the deadline is exhausted. Reject non-finite deadlines before an RPC. - Add a red/green regression and a repeated-gateway-failure case around a real listening process. - Update execution semantics and both recovery maps. ## Verification - Before the fix: the new probe suite failed two behavioral assertions. The gateway case returned its HTTP 502 error instead of the later successful observation. - After the fix: 75 focused probe, diagnostic, process-supervision and adoption tests pass. The diagnostic failure tests use a controlled clock for their full 30-second budget. Their error and redaction assertions stay unchanged. - Map alignment and module-boundary checks pass. - Full workspace typecheck and build pass at final commit `ddd42f66378ebc36ec1d7e0ca1f45943473561b3`. - The broader affected suite against disposable PostgreSQL passed 994 of 997 tests; its three startup/database timeouts passed unchanged in isolation. The initial full default local run encountered embedded PostgreSQL startup failures and did not complete. The four diagnostic tests that assumed the old short retry timing now use a controlled clock without weakening their assertions. - Incident evidence came from Linux AMD64, Node 24.22.2, Core commit `0c70cedce9a803bdc201324528abb35e423e7cd0`, native Codex `gpt-6.1-sol`, and a shared ChatGPT subscription. Two separate runs recorded the same supervision HTTP 502 within one second. - A disposable live Daytona sandbox check passed on Linux x86_64 and Node 24.22.1. Three injected gateway failures recovered in 3.31 seconds on production source from `e54a355fa179022caaa6854302ff56bd85de0229` (unchanged by the final test-only commit). The real Node listening process had one launch, retained its exact identity and saved file across a second read, and produced a confirmed Stop receipt. Sandbox deletion succeeded. This is process-supervision integration evidence, not a Codex task or model execution. - Fixed-revision hosted acceptance and recovery of the original conversation require rollout. This PR does not claim either. The duration of the original provider outage remains unknown. - Final commit `ddd42f66378ebc36ec1d7e0ca1f45943473561b3`: required `ci / verify` and `ci / e2e` checks pass; Greptile is 5/5 with no inline findings. The separate Paperclip Review service has not completed an assessment and remains queued. It is not a required branch check. ## Risks - Persistent fast transport failures cause more read-only requests within the same deadline. Capped backoff limits their rate. - Missing or changed identity still returns immediately. Failed health reads still cannot authorize replacement, process signals, or provider replay. - Signal requests still have one attempt. The timeout budget and execution permissions do not change. - No schema or package changes. ## Model Used - OpenAI GPT-6, through Codex, with reasoning, repository inspection, and shell tool use. The exact deployment model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
1511c92965 |
fix(evals): account for superseded delegation wakes (#18319)
## Thinking Path > - Paperclip lets people ask agents to delegate work and return the result. > - The chat feature eval checks task creation, worker execution, saved output, and delivery to the source chat. > - A task can first belong to the CEO and then move to Morgan. > - Paperclip correctly cancels the old assignment wake before it starts. > - The eval treats that queue record as a failed provider run and stops before Morgan works. > - This change accounts for that record separately and preserves the complete outcome checks. > - The report also keeps partial check results when an attempt fails. ## Linked Issues or Issue Description **What happened?** The delegation journey stopped on an unstarted queue record with `issue_assignee_changed`. The CEO had created a task, assigned Morgan, saved the brief, and returned a task link. The test stopped before worker execution. Its report showed no individual checks even though the saved observation contained partial results. [Original public measurement](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/chat-progress-39e-112-latest-121-measurements-with-screenshots-40649033672/?view=case&case=chat-features.delegate-announcement). **Expected behavior** Keep the obsolete queue record and its events. Continue observing the intended worker. Pass only after three successful native work runs, a current Morgan-authored announcement, a completed task, automatic source-chat delivery, and browser readback. **Steps to reproduce** 1. Ask the CEO to have Morgan write and save a launch announcement, then return it to chat. 2. Create the task under the CEO, then assign Morgan before the queued CEO run starts. 3. Observe the cancelled old assignment. The previous eval aborts at that point. **Paperclip version or commit** Based on `6da112ab40d7b3cc0bcb79e08ddbb9b915a268a4` from `origin/master`. **Deployment mode** Disposable Product E2E fixtures. The live check uses the trusted default-branch workflow and AWS fleet. Related contribution: #17794 introduced these two journeys. The open-PR search found no separate fix for this queue-accounting defect. ## What Changed - Recognize only explicit dispatch-gate reassignment cancellations with a null start time and no native issue. Verify the company, prior CEO owner, and target task now assigned to Morgan before excluding that record from work-run counts. - Keep every queue record in the full observation and final ledger. Retain its events with an explicit reason for the absent provider log. - Let the feature flow own its complete saved-state and browser oracle. Return only actual work runs to the common provider checks. - Record saved-state checks on failures as well as successes. Capture the required `final-state.png` after the feature-specific browser readback. - Add two explicit local OpenCode cells alongside the two existing Codex subscription cells. Use the current production-qualified model. The earlier linked campaign used a different model, so this is a new measurement. - Advance the grader and suite versions. Keep the original request, output requirements, 120-second run cap, five-minute observation bound, budgets, and single-attempt policy. ## Verification - Red/green regressions cover queue accounting, missing-log collection, partial checks on evidence failure, and the required final screenshot. Flow tests invoke the actual polling callbacks: restoring the original callback reproduces both early-stop errors. Started cancellations still fail. - All 87 focused flow/grading/queue tests pass. Catalog/report checks and the 137 additional Node checks pass. Eval typechecking passes on final commit `0a6e11408c4470683417e6bf7965c73cec2a9a60`. - All current CI tests/builds pass. Greptile is 5/5 on the final commit, with no unresolved review threads. - The final commit adds only flow test coverage. Its registered definition digest equals live target `e4e0be875ee12c134fe8dd2ce15a88bfcbb100ef`: `ac6bba21d4b9cb86088be5d00df6442af90268186d6e94c9bea7d611d0488563`. - Replaying the original saved observation still fails for missing Morgan execution and delivery. Its stale queue record no longer counts as provider work. | Operation | Selected coverage | Observed result | | --- | --- | --- | | [41071710105](https://github.com/paperclipai/paperclip/actions/runs/41071710105), `b8dff816` | Backlog and delegation | Refused at the exact private-retention gate before fixtures or providers. Preserved without feature coverage. | | [41073708150](https://github.com/paperclipai/paperclip/actions/runs/41073708150), `39be9e45` | Backlog and delegation | Both passed their full product/browser oracles. Retained grades correctly fail missing `final-state.png`; the feature flow had saved the image under another name. Fixed in `e4e0be87`. Each original archive remains private and intact; later workflow concurrency superseded only the report aggregation. | | [41076854978](https://github.com/paperclipai/paperclip/actions/runs/41076854978), `e4e0be87` | Backlog and delegation | Backlog passed in 92.576 seconds, with a 47.388-second native run, complete required artifacts and passed cleanup. Delegation was interrupted by a runner-host shutdown at 15:19:23 UTC; all later evidence-upload steps were skipped. Its private archive is absent, so behavior, usage and cost remain unverified. | | [41080065077](https://github.com/paperclipai/paperclip/actions/runs/41080065077), `0a6e1140` | Delegation only | Passed on the final commit in 240.786 seconds. All 14 product checks plus browser readback passed; exact private identity, current worker output, Done state, correlated source-chat reply, required screenshot and cleanup verified. The three native runs took 42–60 seconds each. Passing control was not repeated. One-cell workflow and report enforcement passed. | The completed delegation in `41073708150` passed all 14 product checks plus browser readback in 222.593 seconds. Morgan saved the current two-sentence announcement, the task reached Done, and the CEO automatically delivered that saved text to the original chat. Its three successful native runs took 49.117, 40.060 and 39.107 seconds. The failed required screenshot remains a failed result in that measurement. All live operations use explicit IDs, concurrency one, no automatic retries, the unchanged 120-second per-run cap, and scoped private evidence retention bound to the exact workflow, target, original run, selector and launcher digest. These narrow measurements do not qualify the full catalog or establish repeated-run reliability. The broad local support suite passed 5,139 tests, with two skipped. Two database tests hit exhausted host semaphore capacity after the installed Postgres package's missing library symlinks were repaired. Their recheck reported `could not create semaphores: No space left on device`. No host IPC resources or kernel settings were changed. The corresponding full CI tests passed on a fresh host. ## Risks - A broad cancellation exception could hide failed work. This exception requires the dispatch gate's exact receipt, a null start time, the expected company and prior owner, and the actual reassigned task. Started and unproved cancellations still fail. Backlog creation permits no task wake. - New manual OpenCode cells can spend provider money only when explicitly selected. Default scheduled selection is unchanged. - This changes eval interpretation and evidence collection. It does not change product scheduling or task ownership. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact model ID and context-window size are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b31ec0a2dc |
feat(ui): link routines to a filtered task list (#18317)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Routines create tasks that people need to inspect and manage. > - The routines list sends its View all runs link to Audit. > - Tasks already supports routine origin filters in its API, but the page has no control for them. > - This pull request adds a Routine tasks checkbox and links to that selection. > - People can inspect routine work in the normal task list, search, and board views. ## Linked Issues or Issue Description **What existing behavior does this improve?** The routines list link and the Tasks filters. **Current behavior** View all runs opens Audit. People cannot select routine-created tasks in the Tasks filter menu. **Proposed behavior** View all tasks opens `/issues?view=all&originKind=routine_execution`. Routine tasks is selected. People can toggle or clear it from Tasks. **Reason and benefit** People can use normal task controls to inspect work created by routines. The filter includes manually started routine runs. It excludes manual tasks and other automatic work. **Breaking changes** The routines list link changes its label and destination. No API, schema, authorization, or routine execution contract changes. ## What Changed - Add Routine tasks under Visibility in the current, streamlined, and legacy Tasks filters. Keep Live runs only in the same group and use Show agent-created tasks with proper capitalization where that control is available. - Keep the selection in the URL through reload, search, task navigation, and checkbox changes. - Apply the existing API origin filter before pagination. Include routine tasks even when other automatic tasks are hidden. - Keep source filtering in the shared task filter module and collection owners. Reuse the existing company-scoped issue-list contract. Hide the live-run control in the design sandbox, which does not load live-run IDs. - Update regression coverage, product docs, feature/access maps, and the toolbar preview. ## Verification - All 58 final-head checks are complete with no failures for `f46c2e409b20fa372d166f4721601a0471894fb0`, including successful `ci / verify` and `ci / e2e` gates. Greptile and Paperclip Review rate this commit 5/5 with no actionable findings or unresolved review threads. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41078093110). - Local checks pass: full repo typecheck/build for the feature, UI typecheck/build after the final refinements, token gates, feature-map checks, and Storybook catalog generation. - Regression coverage exercises both real list implementations, filter grouping and selection, search and all board columns, URL/pagination state, and mixed pending-decision batches. The Visibility update passed all 89 relevant tests across separate runs. The final sandbox correction passes all 10 filter-menu tests, including three cases that reproduced the bug before the fix. The earlier broad UI run passed 10,043 tests across 845 files. - Browser proof used the actual local app with disposable PostgreSQL data: three tasks created through the routine-run API and one manual task. Followed View all tasks; verified the selected Routine tasks checkbox under Visibility, toggling, reload, search, and board view. Checked current, streamlined, and legacy filter presentations. Screenshots show the link, routine-only results, and requested capitalization/grouping. - Failure accounting: the initial broad UI run found two preference expectations that needed the new false default; those were corrected before the full UI pass. Scrolling tests hit five-second timeouts in combined runs and passed in isolated reruns without timeout changes. A duplicate full local UI run was stopped after the earlier final-commit CI UI lanes passed, with no failures reported in that stopped run. - Full local `pnpm test:run` could not complete because native PostgreSQL initialization exhausted the host semaphore pool. A direct initdb probe confirmed `could not create semaphores: No space left on device`. Browser verification used a disposable container; no host settings or other workspace processes were changed. - Review found that the newly exposed live-run control could empty the design sandbox, which has no live-run IDs. The sandbox now disables that unsupported control; its regression test failed before the fix and passes after it. ## Risks - Low risk. The change uses the existing routine origin and read permissions. A routine task remains automatic work, including when a person starts it manually. - The routine checkbox is exposed only on the Tasks collection. Other filter consumers retain their controls and normalize the new field to false. - Routine histories and Audit run links inside routine details keep their current behavior. ## Model Used - OpenAI GPT-6, through Codex, assisted with implementation and verification. The exact runtime model ID and context window are not exposed in this session. Capabilities used: repository editing, command execution, tests, and browser interaction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |