6165 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
DottaandPaperclip	82c033ed12	test(e2e): cover uploaded input and personal inbox triage (#18643) ... ## Thinking Path > - Paperclip lets people assign work to agents and manage their tasks. > - Product E2E journeys verify the saved outcome through the browser and normal APIs. > - Uploaded input and personal inbox actions need repeatable coverage. > - An agent answer must match the input file, and an inbox action must retain its saved state. > - This change adds two journeys with independent evidence checks. > - A blocked fixture action must use the normal human-owned unblock contract. > - Registration stays separate from deployed qualification and daily admission. ## Linked Issues or Issue Description **What existing behavior does this improve?** Product E2E coverage for file attachments, saved task documents, and personal inbox read and archive actions. **Current behavior** The journey catalog lacks an uploaded-input calculation and a persisted personal inbox triage path. A successful agent run alone cannot prove either outcome. **Proposed behavior** Upload a generated CSV through the task composer. Require an exact saved answer and unchanged input bytes. Read and archive one owned task through the inbox, reload each state, and verify that a new Blocked event brings it back without agent work. ## What Changed - Add a file-input journey with a random reference held only in the uploaded CSV. - Check exact result values, document authorship, native run binding, original input bytes, and persistence after reload. - Add a browser inbox journey with an independently authenticated human author, saved read and archive state, unchanged work status, and activity resurfacing checks. - Reject wrong, missing, duplicate, and foreign evidence in assertion controls. - Register both cases and document their scope and privacy rules. - Supply a current-human unblock owner and action for the archived-task fixture. Test that exact payload through the production route and retain the bare-blocked422 negative control. - Include the fixture constructor in the source definition hash and both new cases in the exact catalog assertion. - Reject collected inbox receipts with unexpected runs, providers, models, or native/legacy execution modes. ## Verification - The frozen source is `cdb6659fb4e4b67e9321bc3fd963cf11e9b8b2f3`. It includes one fixed master integration at `fd0d61b8e329efc89ff541dc2cac44e7c7a37f78`. The single map conflict retains both coverage paragraphs. - The production HTTP route rejects the original bare blocked payload with422 and accepts the exact human-owned fixture payload with200. Neither control wakes an agent. The full related route file passes all200 tests. - The frozen current head passes recursive workspace typecheck, runner typecheck, full build, map checks, and diff checks. Its full related journey/catalog suite passes723 tests, the production route suite passes200, and the separate Node support suite passes347. The retained contract file passes all38 tests. Fresh Greptile review is 5/5 on the frozen current head. Current-head hosted CI passes after one authorized failed-jobs-only recheck. The original Runner close test failed; the unchanged, properly cargo-prepared focused local control passed in2.99s, and the hosted recheck then passed. The cause remains unknown; no Runner code, test timing, or assertion changed. The original hosted diagnostic and the initial invalid local invocation without required fake binaries remain recorded. The ordinary local full-test run is in progress. - The unrestricted local runner support run on the fixed-master integration is not green:5993 tests passed,12 failed,3 skipped. The unchanged failures cover onboarding cleanup expectations, Pi metadata, workflow-security expectations, and an expired test allowance. An untouched743 baseline reproduces the same non-PG failures; their source/test blobs also match pre-feature40b807. This change does not extend an allowance or alter paid admission. The original5983-pass/15-failure run remains recorded. Restoring the local Postgres package's own library aliases resolved its two startup failures. The feature-related catalog omission is corrected. - The uploaded-file case passed its first complete deployed QA attempt: one configured native run, all six independent checks, original evidence, complete cleanup, and no active operations. - The first inbox attempt stopped at the agent comment execution-context guard. That guard stays intact. The fixture uses an approved, independently authenticated QA human. - The next original inbox attempt verified protected-secret retrieval, human attribution, and session closure, then failed its unread prerequisite. The product unread defect was repaired in #18670. - The fresh deployed inbox attempt passed external-human activity, saved read state, archive persistence, and unchanged task status after the product rollout. It then stopped at the fixture's bare blocked PATCH before any resurfacing assertion. The response status was discarded;422 is an explicit contract inference independently reproduced through the production route. That failed attempt remains retained. A fresh full inbox qualification on the current frozen head now passes all six checks and all five phases; the normal human comment receipt, closed human session, zero agent work, complete cleanup, and absence of active operations are verified in the original evidence. Daily admission remains a separate coordinator action. - The retained-result review gap failed a negative receipt control before the repair. All38 retained journey contract tests now pass. The two old unread advisory threads are resolved only after the fresh full deployed inbox pass. - Original failures remain recorded, including the earlier unchanged environment-runtime assertion and unchanged runner/occupied-port startup failures. Closed operation codes and failure screenshots retain the failing step without raw browser errors. Actual retry qualification remains separate from source tests. ## Risks The target must supply the declared external-human fixture. Inbox control selectors may differ across deployed UI revisions. File processing depends on the configured native agent. Exact saved-state and source checks must remain required when those dependencies fail. The change adds test journeys and does not change production behavior or schemas. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (relevant suites; broader baseline failures are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 18:12:16 -06:00
DottaandPaperclip	4814d1ab69	fix(ui): show composer images as rounded thumbnails (#18689) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People use the new-task and thread composers to send text and files. > - Images currently enter the text editor or appear with other files. > - Large images interrupt text entry and make the two composers inconsistent. > - This pull request puts image previews above the text in both composers. > - People can check and remove images while they continue to edit the text. ## Linked Issues or Issue Description Refs: #13206. Related shared-composer work: #15091. **What existing behavior does this improve?** Image selection, paste, and drop in the new-task and thread composers. **Current behavior** Thread images enter the editor at the text cursor. New-task images can appear inline or with file attachments. Images do not have one preview size. **Proposed behavior** Show images above the text as 80 by 80 previews with rounded corners. Use the existing spacing and radius tokens. Keep file attachments in their current row. Add image references when the thread message is sent. **Reason and benefit** People can review images without moving or deleting message text. One shared image row keeps both composers consistent. **Breaking changes** New image uploads stay outside the editor until send. Old inline-image drafts remain readable. No server API or database changes are required. ## What Changed - Add `ComposerImageAttachments` for local previews, removal, upload status, and retry. - Let `TaskChatComposer` own the image row and all image input paths. Keep attachment IDs in the existing send flow. - Let `NewIssueDialog` stage images and upload retained files after task creation. Save file bytes in browser IndexedDB by account and company. Restore them after reload. Clear them after creation. Show storage errors. Merge saved file changes in one IndexedDB transaction. Delete only the files removed by that draft. This keeps files saved by other open tabs. - Keep rich editing for old image Markdown. Use the same MIME and filename checks for image previews in both composers. - Save optional MIME data with thread draft attachments. Existing draft records remain valid. - Add desktop and mobile Storybook examples. Update the task feature map and component tests. ## Verification - `pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/board-attachment-receipts.spec.ts`: all 14 Chromium tests pass against the real local Board, API, storage, and disposable database. The suite proves image recovery in two open tabs after saves and task creation, new-task image recovery after reload, byte-for-byte upload content, cleanup after creation, rich editing of old image drafts, upload receipt recovery, and removal during pending uploads. - The first local browser attempt could not start because Chromium was absent. Chromium was installed. A later run passed 12 tests but the new legacy-draft fixture restored an empty body. That fixture wrote storage while an empty composer was mounted. It now seeds before React startup and uses a real uploaded image. The complete 13-test rerun passes. Early component runs also exposed the new restore timing. Submit and file controls now wait for restoration. The test helper now waits for React effects. - 316 focused tests pass across `TaskChatComposer`, `NewIssueDialog`, `MarkdownEditor`, `composer-draft`, and `useStagedIssueFiles`. The tests cover explicit file removal, file recovery, account and company changes, storage failures, MIME-empty images, upload retries, and removal. - The two-tab browser test failed on the previous commit: the second save erased the first image. The fix merges files in one read/write transaction and records explicit removals. The first fixed run reached the final create step but had no text request after the other tab cleared the shared text draft. The test now enters a fresh request before that step. The full 14-test rerun passes. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, `node scripts/storybook-catalog.mjs`, and `git diff --check` pass. - `pnpm --filter @paperclipai/ui typecheck` and `pnpm --filter @paperclipai/ui build` pass after building the local plugin SDK dependency. Repository typecheck and build stop because this workspace has no `cargo`. Full CI must pass before merge. The first CI run passed typecheck, build, and all unit-test shards. Its browser failure expected the old inline upload label. The assertion now uses the thumbnail upload status. - The prior browser review used the production components with simulated upload and submit calls. Desktop, mobile, and dark-theme captures were accepted before this PR. Browser checks confirmed equal-size rounded previews above the editor, no inline image nodes, no horizontal overflow, and removal that preserves text. These checks do not prove live backend persistence. - Earlier repository typecheck and build attempts also stopped without `cargo`. An earlier full test run was killed with a recorded out-of-memory event during a server integration shard. It had no full-suite verdict. One document-recovery test timed out during that concurrent run. Its isolated rerun and later complete focused runs passed; the initial timeout cause was not established. - An initial catalog command in this workspace ran before dependency installation finished. It could not load Storybook. The command passed after installation. The first UI typecheck could not find plugin SDK declarations while that package was still building. The UI typecheck passed after `pnpm --filter @paperclipai/plugin-sdk build`. To inspect the layout, open `composer-new-task--images`, `composer-new-task--mobile-images`, `composer-add-menu--images`, and `composer-add-menu--mobile-images` in Storybook. Select images with different aspect ratios. Check removal, mixed files, failed upload and retry, and draft restoration. - Greptile reports 5/5 for commit `9b9e93beeed80371f667ad2e1a9573e82332c05c`. There are no open review threads. [CI](https://github.com/paperclipai/paperclip/actions/runs/41544180330) passed on this commit. All 58 checks finished: 54 passed, one review is neutral, and three optional checks were skipped. The legacy Snyk status passed. Paperclip also reports 5/5. No review threads are open. ## Risks - Open tabs share the account and company draft. Atomic file writes keep changes from other tabs. A reload can restore those saved files. - Attachment state now controls when image references enter a thread message. Tests cover pending uploads, failures, retry, removed images, and retained upload IDs. - New-task images use the existing staged-file upload flow after task creation. A failed upload follows that flow's existing error behavior. Browser storage can be unavailable or full. The composer keeps the files in memory and shows a recovery message. - Legacy inline-image drafts retain their original text. No migration is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, tool use, and shell execution. The exact runtime snapshot ID and context-window size are not exposed in this session. The earlier implementation's model snapshot is not recorded in the handoff. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details available in this session) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My remote branch name describes the change and contains no internal Paperclip ticket ID - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 23:35:32 +00:00
sentry[bot]	18abf7c905	fix(db): tolerate unparseable connection URLs during teardown (#18684) ... closeRegisteredClients now uses hostPortKeyOrNull and returns early for URLs that new URL() rejects, matching createDb, which never registers clients under such URLs. Fixes PAPERCLIP-BACKEND-5C Fixes #18685	2026-11-02 15:33:36 -08:00
Nicky Leach	3a7bb4dac6	perf(runner): activate shared retained startup context (#18405)	2026-11-02 15:12:57 -08:00
DottaandPaperclip	58308a3e63	perf(apps): scope sidebar review counts to one connection (#18688) ... ## Thinking Path > - Paperclip helps people manage AI agents and their governed app access. > - The Apps sidebar displays a Review badge for one connection. > - It polls the full company attention service every 30 seconds to get two counts. > - That service also loads every profile, rule and active catalog entry. > - This pull request adds an authorized connection read for only those counts. > - The same badge now avoids unrelated profile evaluation and database result decoding. ## Linked Issues or Issue Description **Current behavior** The contextual sidebar polls company-wide attention. It needs only one connection's signed pending action count and whether any catalog entries are quarantined. **Proposed behavior** Both sidebar variants poll a connection-scoped count endpoint with existing connection read authority. The full attention API stays unchanged. **Reason and benefit** The retained 200-profile fixture performs seven service queries and returns 80,115 rows for this badge. The new reader performs one query and returns one row. This reduces local service CPU and avoids unnecessary decoded profile/rule data. Related merged evaluator fixes are already in the baseline: Refs #18673, Refs #18681. A search found no existing PR for this scoped count reader. ROADMAP.md contains no duplicate planned work. ## What Changed - Add the narrow shared DTO, typed client, guarded REST/OpenAPI route and catalog-owned reader. - Count independent aggregates with both company boundaries and exact signed-argument semantics. Keep archived counts at zero. - Update both contextual sidebar variants. Keep the exact badge, 30 second interval and existing attention-prefix invalidation. - Add PostgreSQL differential tests, authenticated HTTP tests, sidebar tests, a repeatable benchmark and access documentation. ## Verification - Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture refinement: `230c42ee15`; final local existence gate: `390a218254`. - Passed: focused PostgreSQL differential test; assembled HTTP test with board session and agent JWT; 14 sidebar component tests; token gates; map checks; diff check. - Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed references, 12 agents, a sibling connection and 2,000 unrelated historical invocations. Complete normalized two-count results are equal on both paths. - Quiet local measurement: seven alternating warm pairs after two warmup pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new service CPU median: 213.891/1.680 ms; ranges: 209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges: 151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1. - Run `cd server && node --expose-gc --import ./node_modules/tsx/dist/loader.mjs scripts/benchmark-connection-review-counts.ts`. See `doc/connection-review-counts.md` for scope and failure history. Byte-volume accounting runs outside timing; no HTTP payload reduction is claimed. - Initial raw scalar SQL failed with ambiguous columns. Typed subqueries fixed it and passed real PostgreSQL equality. Initial fixture agents lacked permissions and correctly received 403. The final fixture grants explicit authority and tests revocation. An initial benchmark included byte serialization in timing; the reported run excludes it. - Passed required local `pnpm -r typecheck` and `pnpm build`. The initial sandbox typecheck could not open the tsx IPC socket; the escalated rerun passed. Full monolithic `pnpm test:run` was not run locally: repeating a 15–25 minute suite serially across both performance PRs adds no distinct evidence. Broad suite acceptance uses final-head CI, which runs the same general/serialized test partitions and the native Runner lane. Final-head CI is in progress. - Passed built-static Chromium journey on an owned local-trusted server and disposable PostgreSQL: Review 3 appears, only scoped counts are requested by this sidebar, resolving requests/clearing quarantines then reloading removes the badge, and archived counts are zero. The first ad hoc launcher omitted required decision-service options and failed at startup; using the normal no-op wake callback fixed that harness. - Greptile identified an invalid resolved-request fixture label. Corrected `denied` to the declared `rejected`, then reran the real PostgreSQL differential test. The count SQL and non-pending exclusion are unchanged. - The first final-head CI failed the static cross-tenant existence-oracle guard because the new explicit resource company assertion had no nearby `hasCompanyAccess` gate. Existing management admission already rejects foreign resources with 404. The route now also follows the standard local two-step gate. The unchanged guard plus authenticated HTTP negative cases passed (3 tests). No allowlist or test was weakened. - These local service measurements do not prove hosted performance. No deployment is included. ## Risks Low risk. New read admission reuses existing current-authority checks. The old attention API is unchanged. Counts still require database work over relevant pending/catalog rows. No cache, schema migration or telemetry is added. Roll back this PR if the badge differs or scoped reads deny previously permitted users. ## Model Used OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code execution. The host does not expose the configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 17:12:22 -06:00
commitperclip[bot]	fd0d61b8e3	fix: Compare native finaliser lease expiry at millisecond precision when releasing an expired owner (#17214) ... > [!NOTE] > Opened by Prospector's automated issue-fix pipeline. A maintainer reviews and decides; nothing here merges on its own. ## Thinking Path - Issue #16715 reports a defect in paperclipai/paperclip. - Root cause, as the fixing agent read it: Lease renewal writes 'lease_expires_at' from the database clock ('now() + interval'), so the stored value has microseconds. 'agentExecutionsHaveStopped' reads that value into a millisecond JS 'Date' and then releases the row with an exact 'eq' on it. The update never matches, so the stop check returns false on every tick and the agent stays in 'pausing'. - This pull request: Compare native finaliser lease expiry at millisecond precision when releasing an expired owner ## What Changed - `server/src/services/agent-execution-stop.ts`: The release update now truncates the stored lease expiry to milliseconds before comparing it with the value it read, using the same 'date_trunc('milliseconds', ...) = ...::timestamptz' pattern as 'chat-channels/activity.ts'. The other guards stay the same: owner, generation, phase and the null result. This was the only exact-match comparison on a lease expiry in 'server/src'. ## Verification - Reproducing test(s), added in this PR: `server/src/modules/agent-lifecycle/__tests__/lifecycle.test.ts` - They fail on the base commit and pass with the fix, each twice in a fresh sandbox. - 3 independent attempts wrote 3 reproductions; this fix passes every one of them. - The repository's compile command passes with the fix applied. - The full test suite passes except for the 10 file(s) that already fail on the base commit. ## Risks - Written by an automated pipeline, not a person: review it as you would an outside contribution. - Risk tier of the touched paths: 2. - The scope reviewer found it also changes, beyond the report: agentExecutionsHaveStopped for an expired finaliser row whose lease_expires_at is already millisecond-precise (written from a JS Date) - The scope reviewer found it also changes, beyond the report: Finaliser row whose lease was renewed or taken over between the select and the update - The scope reviewer found it also changes, beyond the report: Rows with null leaseExpiresAt ## Model Used - codex:gpt-6.1-sol, opus, sonnet (Anthropic Claude, via Claude Code; OpenAI, via the Codex CLI), writing and checking the change in an isolated sandbox. - opus (Anthropic Claude, via Claude Code), reviewing it independently. ## Linked Issues Fixes #16715 Other pull requests that name #16715: - #17204 (open): fix(heartbeat): stop loading full run rows and stale pids when pausing an agent - #16745 (open): fix(agent-lifecycle): release an expired native finalizer lease at millisecond precision ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with 'Fixes: #' / 'Closes #' / 'Refs #' OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub '#NNN' / 'github.com/paperclipai/paperclip' URLs) - [ ] My branch name describes the change (e.g. 'docs/...', 'fix/...') and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge <!-- prospector:issue-fix v1 issue=16715 base=c7ea5c0216b8 report=1a4e13ecd7c6cdb6 --> Co-authored-by: commitperclip[bot] <284812584+commitperclip[bot]@users.noreply.github.com>	2026-11-02 15:01:52 -08:00
DottaandPaperclip	554a770c6f	perf(apps): scope connection profile configuration reads (#18687) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Apps Permissions page reads saved access profiles for one connection. > - The page fetched every company profile, rule, and binding. > - It used only the app profile and eligible task-granted profiles for that connection. > - This pull request adds a saved configuration read owned by the profile service. > - The route keeps current connection and per-profile management checks. > - The benefit is fewer database rows, less Node work, and a smaller response for the same displayed configuration. ## Linked Issues or Issue Description **Subsystem affected** Server profile configuration reads and the Apps Permissions and Review data caller. **Problem or motivation** One connection page loaded and serialized unrelated company profiles. In the local fixture, this produced a 32.35 MB response with 76,001 rule rows. **Proposed solution** Select the displayed connection's app profile and exact eligible connection-intent profiles before loading their entries and bindings. Return an explicit configuration DTO with complete selected arrays. Keep summary and pending-review work on the existing full API. **Alternatives considered** Filtering the full response in the browser retains the database and Node cost. Truncating the full API would break other clients. Filtering entries to the connection would lose selectors and bindings needed by saves. **Roadmap alignment** This improves the existing governed Apps capability. It does not add a new access policy. Related merged work: #18673 and #18681. Related harness cleanup: #18683. Public PR searches found no duplicate connection configuration projection. ## What Changed - Add `ToolProfileConfiguration` and `GET /api/tool-connections/{connectionId}/profile-configuration`. - Select `app:<connectionId>` at any status and exact active, correctly bound `connection_intent` profiles. Run the existing per-profile visibility filter before configuration queries. - Preserve complete ordered entries and bindings. Keep all save behavior, empty-binding install fallback, and the existing full profile API. - Switch AppDetail and AgentConnectionAccess to the configuration DTO. Use a distinct cache key below the existing profiles prefix so mutation and reconnect invalidations still refresh it. - Add real PostgreSQL differential and authenticated HTTP tests, UI caller coverage, a reproducible benchmark, and access-map documentation. ## Verification Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, after both #18673 and #18681. Final head: `b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`. Current implementation and reproduction details are in [the performance note](https://github.com/paperclipai/paperclip/blob/b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22/doc/performance/2026-11-02-connection-profile-configuration.md). - AppDetail: 79 tests passed. Existing access/action saves, install fallback, grant removal, errors, and refresh cases use the new caller. - Real PostgreSQL differential: passed. Full ordered selected configuration equals the old full-list selection. Cases cover extra bindings, sibling selectors, wrong source/key/status/binding, null and non-string agent metadata, fresh mutations, empty app bindings, visibility filtering, and company scope. - Production HTTP authentication: API key and run JWT cases passed. Hidden personal accounts omit spanning profiles. Private connection reads fail. Restored ownership restores visibility. Revoked management grants deny fresh calls. Final UID canonicalization and foreign-company 404 assertions passed on the current head. - Map structural check, token gate, and `git diff --check`: passed. - Required local `pnpm -r typecheck` and `pnpm build`: passed on the current head. `pnpm test:run` was started once. Its first chat shard passed (337 tests, 151.87 seconds); the second shard was intentionally interrupted after the complete final-head CI suite passed. This partial local attempt is not a full local test pass. The same stable-runner general and serialized partitions and Runner lanes passed in [final-head CI](https://github.com/paperclipai/paperclip/actions/runs/41538805231), including aggregate `ci / verify`. No local assertion failures were observed before interruption. - Final-head focused acceptance: four files, 85 tests passed. Full token gates, maps, and module boundaries passed. - Actual browser acceptance passed on the current checkout with isolated real PostgreSQL and a local trusted board. An action saved as Off persisted after reload. Both app agent bindings stayed intact. Removing an agent-only task grant persisted after reload. Requests used only the narrow profile endpoint. No browser exceptions occurred. No responses were mocked. No provider calls or hosted changes were made. The helper stopped the owned runtime and database after the check. - Final exact-head checks: 54 success, one clean neutral Paperclip Review, three skipped, zero pending or failures. Legacy Snyk status: success. No inline review comments. - Greptile reviewed exact head `b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`: 5/5, no findings, completed successful check. Five alternating unprofiled pairs ran with real PostgreSQL on macOS/arm64, Node v24.22.2, and forced GC. The fixture has 210 profiles (200 heavy profiles plus ten eligibility cases), 76,001 rules, 1,900 active tools, 695 removed tools, and 12 agents. The selected configuration has two profiles and 1,391 rules. Each pair asserts complete ordered selected DTO equality. | Measurement | Full list | Configuration read | | --- | --- | --- | | Service wall ms, median [min, max] | 200.927 [193.319, 213.390] | 6.414 [5.508, 6.973] | | Node CPU ms, median [min, max] | 262.751 [246.727, 271.573] | 27.525 [22.576, 29.308] | | Select queries / returned rows | 7 / 78,139 | 3 / 1,397 | | Actual response JSON bytes | 32,353,534 | 592,233 | | Serialization wall ms, median [min, max] | 90.714 [88.715, 99.304] | 1.644 [1.637, 1.745] | | Serialization CPU ms, median [min, max] | 90.644 [88.569, 96.778] | 1.654 [1.637, 1.807] | Reproduce: `node --expose-gc --import ./server/node_modules/tsx/dist/loader.mjs server/scripts/benchmark-connection-profile-configuration.ts 5`. The benchmark measures the service and actual response JSON separately. It excludes HTTP and authorization work through an identity visibility callback. Node CPU does not include PostgreSQL CPU. No hosted or foreground-under-load improvement is claimed. Preserved failures: offline install lacked a locked tarball; frozen online install passed. Sandbox localhost and tsx IPC restrictions blocked initial DB/typecheck attempts; permitted retries use the same checks. The first HTTP suite needed the unbuilt plugin SDK; its owning build helper fixed setup. Shared typecheck caught a missing barrel export, now fixed. A broad typecheck inherited the host's unpinned nested pnpm executable; the owned stalled process was stopped and restarted with pinned pnpm 9. The complete current-head CI suite and aggregate verify passed. The separate Paperclip agent review published 5/5 with no findings at this head and completed with a clean neutral conclusion. An added foreign-company assertion expected 403; the existing guard correctly returned 404. Only the test expectation changed. The first real browser attempt saved and reloaded the action correctly, but grant deletion returned 422: the stress fixture had a company binding, which makes that profile the company default. The UI showed the existing deletion error. The second browser seed used the production agent-only intent shape and passed removal and reload. This does not change the stress fixture or product deletion gates. Initial external browser script attempts also needed an ESM extension and the correct dependency path. Logs, first-failure screenshots, traces, and result files are retained in the local evidence directory. ## Risks - SQL selection must stay aligned with connection-intent eligibility. Differential fixtures cover the exact current predicate. - A selected connection can still have many rules. This change preserves all selected configuration. - Per-profile authorization still reads current selected resources and can add work. Connection visibility does not bypass that check. - No migration, default-allow, effective-access, policy-review, provider, or deployment behavior changes. The old endpoint remains available. Rollback restores the old AppDetail caller and removes the additive endpoint. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and repository tools. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:55:56 -06:00
DottaandPaperclip	00fea0da52	refactor(perf): reuse database test setup for profile benchmark (#18683) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool profiles govern the tools that agents can use. > - Profile listing and app attention shared nested selector scans that consumed the server CPU. > - #18681 and #18673 replaced those scans with indexes for each computation. > - Their performance benchmarks need consistent setup and reproducible per-service measurements. > - This pull request uses the shared helper entry point and retains the exact measurement harness and results. > - Reviewers can inspect both the CPU savings and the limits of the evidence. ## Linked Issues or Issue Description Refs #18681 and #18673. **What existing behavior does this improve?** The consistency of benchmark setup and the reproducibility of evidence for the two merged CPU fixes. **Current behavior** The profile-review benchmark imports the database starter directly. The matching benchmark uses the shared server test-helper entry point. On the current base, that helper re-exports the database starter, which already supports `PAPERCLIP_TEST_POSTGRES_ADMIN_URL`. The previous committed benchmark measures both services together; the per-service measurements and separate HTTP/Inspector runs lacked a committed harness. **Proposed behavior** Use the same test-helper entry point in both benchmarks. Retain the compile, per-service timing, Inspector, and HTTP-probe programs with their exact commands and raw samples. Preserve the existing embedded and admin-server database behavior. **Reason and benefit** A shared entry point keeps benchmark setup consistent. The committed harness lets reviewers reproduce and compare each real owning service before and after the merged fixes. The failure-path checks verify that result-write errors still release the database and probe resources. **Breaking changes** None. This change affects benchmark setup and documentation. ## What Changed - Import the database starter from the repository test helper in `benchmark-profile-review-projection.ts`. - Record separate CPU and elapsed measurements for profile listing and app attention. - Retain the actual compile, per-service measurement, Inspector capture, and separate-process HTTP probe harness. - Always tear down the test database, HTTP server, and probe even when result writing fails. - Retain all timing and responsiveness samples as sanitized JSON. - State the runtime, fixture, source revisions, and limits of the local comparison. ## Verification - 24 focused tests passed across selector matching, pending projection, review-service freshness, and effective assignments. - Four selected real database-backed service/route tests passed for summaries, review changes, auto-allow behavior, and company isolation. The other 494 tests in that file were not selected. - Injected an `EISDIR` result-write failure into both profiling programs on an owned local admin server. Each child exited with the expected error and removed its disposable database. The responsiveness child also terminated its probe/server handles. - The changed benchmark completed with a configured local PostgreSQL admin URL. Its full responses matched the baseline, and query/row counts matched. - The exact documented compile, benchmark, and responsiveness commands completed from the committed harness at `ca458baddc6461fb6c888984246e329e2bb4bdfe`. The JSON records that build revision and the four owner source hashes. - Five alternating samples per service and implementation used Node 24.22.2 with compiled ES modules. No runtime TypeScript loader ran in those measurements. - Median process CPU from the committed harness: profile listing 3,164 ms to 385 ms; app attention 2,466 ms to 221 ms. Both versions returned identical ordered results and made seven queries per service. - Separate Inspector profiles confirmed that the repeated scope and summary scans were removed. Three separate-process HTTP probe trials per service showed less event-loop blocking. These are local synthetic service results, not hosted page-load acceptance. - Initial local setup failed on missing embedded PostgreSQL library aliases and an unbuilt plugin SDK. Restoring local aliases and building the SDK resolved those setup failures. The original failed setup logs were retained; the 28 selected tests then passed. - The merged summary fix CI run [41535304968](https://github.com/paperclipai/paperclip/actions/runs/41535304968) hit two 15-second timeouts in the unchanged attention-retention history cases. The server shard passed unchanged on one failed-jobs-only retry. Original and retry logs are retained. The compared runs used different runners and synthetic merge trees, so the original timeout cause is not established. No timeout or assertion was changed. - Full repository typecheck, test matrix, and build passed in CI on final head `2f484ed4284f3fa632d8e73d9be084827d1c987e`: 54 successful check runs, three expected skips, one non-blocking Paperclip review, and a successful Snyk status. No failures or pending checks remain. Those broad commands were not rerun locally for this benchmark-tooling follow-up; the 28 selected local regressions and the actual profiling/failure-path programs ran locally. - Fresh Greptile is 5/5 on that exact head. Both review P2 threads are resolved. The Paperclip review's current-base documentation note is corrected in this description; its optional tooling suggestions remain outside this bounded change. ## Risks - The test helper creates a unique disposable database on the configured admin server and removes it afterward. The operator must use a database server where those test operations are appropriate. - Synthetic data and shared-host activity affect absolute timings. The measurements do not establish a production latency percentage or capacity limit. - Production behavior is covered by the two linked merged fixes. This follow-up changes only benchmark setup and documentation. ## Model Used OpenAI GPT-6 through Codex, with code execution and repository tools. The exact runtime model revision and context window were not exposed in this session. The original linked performance fixes record their own model details. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:50:58 -06:00
DottaandPaperclip	3f8ebfff8a	fix(evals): pass named environment secrets to case workflows (#18680) ... ## Thinking Path > - Paperclip lets people manage AI agents and inspect their work. > - The chat catalog sends each original Product E2E result to TestOps. > - Each case uses a reusable workflow with separate protected job environments. > - The caller passed literal empty strings for three secret names. > - A live worker received an empty model key although its environment had that secret. > - GitHub requires matching named secret references at the caller boundary. > - This PR uses those references and keeps the closed credential boundaries. ## Linked Issues or Issue Description Refs #18647, #18666, #18676. **What happened?** The first two workers in a live streaming check failed before a qualified native case. Their model-key environment values were empty. Name-only metadata confirmed that the key exists in the intended protected environment. Both closed diagnostics recorded zero runs, zero checks and no original result carrier. The remaining case was cancelled before its original execution. These records are setup failures, not feature or model grades. **Expected behavior** Each caller passes only the three declared secret names through matching expressions. Each called job uses its own protected environment and only its existing credential consumers. GitHub documents this requirement even when the secret exists only in that environment: [Reuse workflows](https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows). **Steps to reproduce** Run the new `paidWorker` consumer control against the parent helper. It rejects the supported named-reference caller. The deployed literal-empty caller also has the retained live empty-key witness. ## What Changed - Replace six literal empty values in the two case callers with exact matching secret references. - Keep the host-contract resolver closed to those three names and expressions. - Reject blank, extra, mismatched and inherited credentials in focused consumer and workflow controls. Keep the existing lane and authorization controls. - Keep the called workflow, protected environments and credential-consuming steps unchanged. No secret was read, copied, rotated or added during this repair. ## Verification - Parent consumer control: RED with `Workflow caller boundary mismatch` for the named-reference contract. - `node --test tests/runner-e2e/chat-completion-testops-contract.test.mjs tests/runner-e2e/chat-catalog-testops-streaming.test.mjs`: 36 controls pass. - Review found an existing streaming-test mutation that became a no-op after the caller contract changed. Its actual `Missing expected exception` failure is retained. The mutation now uses a blank value, and its description names the current contract. All production posts are unchanged by this test correction. - The focused `Runner eval workflows pin actions and gate paid live execution` security control passes. - Four JavaScript syntax checks, actionlint for caller and callee, and `git diff --check` pass. - An independent source peer checked the four original frozen posts and unchanged callee. The two-line dependent-test correction has a separate peer review. - Local checks use synthetic values, source files and mocked boundaries. No local app, database, native build or provider ran. - Corrected-head GitHub CI passes with 48 successful jobs and one skipped. Both required aggregates, typecheck, build and browser checks pass. This head needed no CI recovery. - Live nonempty resolution and incremental TestOps publication still need qualification after delivery. ## Risks - The previous assumption that literal empty placeholders would resolve environment secrets was wrong in the observed run. This change follows the current primary contract. Local controls do not prove GitHub runtime resolution. - The three names stay explicit. There is no `secrets: inherit`, new secret, environment change or permission expansion. The model worker, private reader and TestOps uploader keep their existing separate consumers. - The immutable evaluation source, cases, original grades, run limits, deadlines and zero-retry policy do not change. Earlier setup failures remain recorded. The existing large batch is not changed or replayed. - Streaming publication and final backend coverage remain unqualified until a real small run proves them. ## Model Used OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does not expose a context-window size. The independent source peer uses the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.7	2026-11-02 16:25:55 -06:00
DottaandPaperclip	d0ffa179b4	perf(tools): index profile selectors per resolution (#18673) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Core resolves tool profiles before it prepares runtime MCP assignments. > - A selected profile can contain a rule for each individual tool. > - The resolver and profile summary scan these rules for each catalog row. > - This repeats millions of exact comparisons for a large selected profile. > - This pull request indexes each selector kind once per computation. > - The benefit is less server CPU with the same current access decisions and result contract. ## Linked Issues or Issue Description **What existing behavior does this improve?** Effective tool assignment and profile summary computation for large selected profiles. **Current behavior** Each catalog row scans the profile's includes and excludes. The work grows with catalog size times rule count. The runtime assignment path also computes a profile summary with the same scans. **Proposed behavior** Build five selector sets for each include or exclude group. Match catalog rows with exact set membership. Retain all current database reads and the full result contract. **Reason and benefit** Reduce repeated JavaScript comparisons. On the synthetic PostgreSQL fixture below, median process CPU per resolution falls from 50.068 ms to 10.129 ms. This is independent of [#18371](https://github.com/paperclipai/paperclip/pull/18371), which removes unused administrative results and narrows instruction descriptors. This PR retains those results and does not include that change. ## What Changed - Index application, connection, catalog-entry, tool-name, and risk selectors in the profile owner. - Use indexed matching in effective access resolution and profile summaries. - Add a differential test against the original single-rule matcher. Cover null applications, exact special-character names, unknown selectors, and empty rule groups. - Add a PostgreSQL regression for per-profile exclusion precedence, the union of multiple effective profiles, and current rule removal. - Add a real PostgreSQL resolver benchmark and document reproduction and limits. ## Verification Baseline: `410b0e27015495ab8951768b5b0b94727de33785`. Treatment: `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`. Node 24.22.2 on macOS arm64. Source modules run with `tsx`; these are local service measurements, not production-build or hosted acceptance. The real owning resolver ran against disposable PostgreSQL with 1,890 active catalog tools, 1,575 individual include/exclude rules, one deny-by-default effective profile, and 12 agents. The installation dependency returns an empty list because the fixture has no installed connections. There are no provider calls. Full returned assignments, profile summaries, rules, bindings, and names are identical to baseline. After two warmups per version, seven alternating samples ran ten resolutions each. Other workers paused CPU measurements and builds. Unrelated host work was not controlled. | Per resolution | Baseline median (range) | Change median (range) | | --- | --- | --- | | Process CPU | 50.068 ms (46.380–58.880) | 10.129 ms (8.089–11.072) | | Elapsed time | 50.362 ms (44.799–106.408) | 8.853 ms (7.719–18.590) | | Queries | 6 | 6 | | Returned database rows | 3,480 | 3,480 | Median process CPU falls 79.8%; median elapsed time falls 82.4%. The long baseline elapsed sample had no matching process CPU increase. Query and row counts do not decrease. Small or broadly allowed profiles can save less. This does not establish whole-run latency or hosted performance. Reproduce with `server/scripts/benchmark-tool-profile-matching.ts`; baseline module preparation is in `doc/mcp-discovery-performance.md`. Passed on the treatment source: ```sh node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-matching.test.ts server/src/__tests__/tool-effective-assignments.test.ts server/src/services/native-runtime/runtime-context.test.ts server/src/__tests__/heartbeat-runtime-mcp-servers.test.ts ``` All 45 tests in four suites passed. `git diff --check` passed. The first PostgreSQL test attempt failed at setup because the sandbox blocked its loopback listener (`EPERM`); the authorized unsandboxed retry passed. No behavior failure was discarded. At measured head `895ee2eec15cf4cdedee4c257fcdddb2a7000a1b`, CI typecheck, build, canary, Runner checks, and all eight browser shards passed. Server general shard 8/12 failed an unchanged background-quietness assertion in `heartbeat-run-dispatch-work.test.ts`: expected 235 select calls, observed 236 after advancing fake timers. Its cause remains unverified; no baseline reproduction was run. That head also failed the module-boundary policy because the benchmark inserted synthetic agents outside the test fixture tree. Follow-up head `e20165b6eb` moves the fixture body byte-for-byte into `server/src/__tests__/helpers/tool-profile-matching-fixture.ts`, following the existing listing benchmark pattern. The production matcher and benchmark measurement loop are unchanged. `node scripts/check-module-boundaries.mjs`, a Node 24/tsx fixture import check, and `git diff --check` pass. The benchmark was not rerun for this extraction. New-head CI and review are pending. Greptile gave the measured head 5/5 with no actionable issues. No deployment or live runtime mutation was performed. ## Risks Exact equality must remain identical to the original matcher, especially for null application selectors and exclusions. The differential and PostgreSQL tests cover these cases. Sets add linear temporary memory in rule count. They live only for the current computation. Current company-scoped state reads, profile precedence, revocation, and runtime assignment digest checks remain in place. Revert this commit to roll back. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code execution, tests, and local PostgreSQL benchmarking. The context window size is not available in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:13:18 -06:00
Nicky LeachandPaperclip	a9b18e1cbe	fix(runner): preserve cleanup ownership across startup failures (#18658) ... ## Thinking Path > - Paperclip controls which run owns an agent task. > - A plugin can continue accepted work after Core stops waiting. > - A late task-creation reply can therefore arrive after startup cancellation. > - Core must keep that work owned until its resource identity and cleanup outcome are known. > - This PR supplies the cleanup and recovery support before shared startup is enabled. > - The dependent activation PR can then reuse transport data within a bounded startup context. ## Linked Issues or Issue Description Refs #18405. This is the independently mergeable prerequisite for that stacked activation PR. **What existing behavior does this improve?** Cleanup of explicitly recorded startup work that outlives its caller. **Current behavior** Caller timeout, worker exit, or failed cleanup-intent persistence can leave work without a complete recovery path. **Proposed behavior** Retain the cleanup obligation, use existing Stop and recovery queues, and release ownership only after the exact task and accepted work have stopped. Ordinary calls keep their existing startup payloads and budgets. This PR does not create startup contexts or advertise transport reuse. **Reason and benefit** Separate the failure-handling contract from activation so each can be reviewed on its own. This support PR has **no standalone TTFT speedup**. ## What Changed - Add the startup cleanup marker, exact ownership checks, verified completion, and acknowledgement helpers. - Connect recorded cleanup obligations to existing Stop, deadline reconciliation, restart recovery, and retry scheduling. - Retain typed cleanup responsibility when a database write fails. Keep the nonterminal run from being treated as ordinary completed cleanup. - Observe late acquisition results and save resource identities before retiring them. Retain real host writes and received results across worker exit or persistence failure. - Queue abandoned replacement tasks through the existing creation journal and cleanup machinery. Require the replacement's own Stop proof. - Accept an explicit startup context in the internal bridge for lifetime tracking. Automatic context creation, marker production, capability advertisement, transport capture, and executor activation belong to #18405. - Add PostgreSQL and worker IPC regressions, including legacy payload assertions. ## Verification Initial split source `8fb482d8be9e70101b0cfdd0830f41cf5b8e3cf4` had tree `95508bd1859e991670cad8afd4de8d2167009ac5`. **190 distinct targeted tests pass**, along with server typechecking. The selected checks cover real PostgreSQL cleanup/ownership, worker IPC, the bridge and creation journal, ordinary/distributed Stop, restart and deadline recovery, idle-sleep behavior, and the unchanged legacy warm-task suite. Independent recovery and worker reviews found no blocking issue. The first correction, `b6342d42042601b376da38dc8e1dba9ca80e9539`, fixes the replacement cleanup finding with the existing startup-cleanup marker, scoped to contextual abandoned task creations. The replacement was never prepared, so preparation-stop proof is not required. No dispatch is fabricated and no normal-startup query is added. The actual queue/sweep/driver/receipt/cancellation regression fails before the fix and passes after it: void, running or foreign-task status retains ownership; exact-task stopped status permits completion even when Release returns void. **64 affected targeted cases and server typechecking pass**. Independent review found no blocker. Current source is `f53af72af76f263096c3c2e788506e6faef9474f`. It fixes both recovery findings from the prior head's Apex 3/5: orphan recovery now preserves pending or completed-but-not-cancelled cleanup through its existing final-write fence, and exact late identities can reach verified cleanup after reconciliation or restart. The regression selection went from four failing assertions to six passing cases. Affected selections cover 217 distinct cases; the exact final six-case regression and server typecheck pass. Independent reviewers found no blocker in either fix. Current-head standard review is 5/5 and all CI checks pass (54 successes, three expected skips, plus Snyk). Apex also completed at **5/5** on this exact head, with zero unresolved threads. The requested independent availability audit is complete on this head. It found an activation-dependent operability gap: an acquisition response that is never received can leave an unknown creation identity with no supported task inspection/reconciliation path, while cleanup keeps the run owner reserved. This does not occur in ordinary lower-only calls. The contract and recovery policy need follow-up before activation; the external review scores do not resolve that availability tradeoff. The split preserves the accepted implementation at `21db7b3851` after its clean merge with frozen master `946735d03a55c1fb4cc5434d6bf1d87742daa9df`. The initial dependent activation PR restored exact aggregate tree `f68c6895f87c7b93ae03e6324d955d2659a99dce`. Both subsequent cleanup corrections are integrated into upper head `27978f294a02da3992585c7f4daa73cd8fcf99f8`; its current aggregate tree is `a0b66c33f5e22745f034f45d24d213f5c210cf16`. Lower-only payload tests cover the absence of capability advertisement and transport projection. Only targeted local tests and typechecking are used. Full regression coverage runs in GitHub CI. No deployment or ROM qualification is part of this change. The lower audit covered connected worker, creation-journal, Stop, restart, deadline and orphan-recovery paths. A new isolated real-PostgreSQL characterization reproduced the unknown-ID hold through actual recovery, cleanup verification and run-control cancellation. It models a lost acquisition response; it is not a live provider or complete IPC test. No source edits were made by the audit. The upper/composition audit remains required after its review cycle. ## Risks - Recording a replacement result now commits with the existing cleanup notification. This conditional replacement path adds transaction/notification responses; common retained startup is unchanged. - This adds cleanup and recovery state handling. A missing receipt or permanently inaccessible old backend can keep ownership held indefinitely. Temporary outages retry; an ordinary retry is not evidence that a deleted or replaced backend stopped. - A caller timeout does not undo work already accepted by a plugin. Unknown resource creation remains unresolved until its identity/outcome is established. Current task creation has no supported inspect-by-operation or operator reconciliation API for a permanently lost identity; repeated Stop alone cannot resolve it. - The support code understands records produced by the dependent activation PR. It does not enable the shared startup policy by itself. - Existing legacy callers must retain their behavior; context-only automatic replacement cleanup remains explicitly scoped. ## Model Used OpenAI Codex, GPT-6 family. The exact model ID and context window are not exposed in this session. Tool-assisted implementation, targeted testing, and independent subagent review. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, with available version and capability details - [x] I have checked ROADMAP.md and confirmed this is the requested split of existing work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue with the required fields - [x] I have not referenced internal issues or private links - [x] My branch name describes the change and contains no internal ticket ID - [x] Targeted local tests pass on this source - [x] I have added or updated relevant tests - [x] I have updated relevant documentation - [x] I have documented the risks - [x] All current-head GitHub CI gates are green - [x] Current-head standard Greptile and Apex are 5/5 with no unresolved findings - [x] I will address all review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 14:13:00 -08:00
DottaandPaperclip	b4315943e8	fix(heartbeat): keep remote retry workspace strategy stable (#18660) ... ## Thinking Path > - Paperclip preserves agent work across separate execution services. > - A remote retry can create a new isolated Git workspace. > - Later admissions compare its saved configuration with the current policy. > - Fresh preparation also creates a temporary controller-side strategy. > - Remote preparation saved that temporary strategy in its fingerprint. > - The next unchanged turn then reported a workspace strategy replacement. > - This change saves the admitted remote strategy so unchanged reuse stays stable. ## Linked Issues or Issue Description Related preparation work: #18543 and #17924. I searched open and merged workspace/session repairs and public workspace-strategy issues. None covers this fresh remote retry case. **What happened?** After Retry created an isolated remote Git workspace, each unchanged follow-up reported `workspace_config_freshness` with `action: replace` and `changedCategories: [strategy]`. It retained the same workspace. The stored fingerprint contained `project_primary`, which was only a temporary controller provisioning choice. **Expected behavior** Save the configured remote strategy. An unchanged follow-up must reuse the same workspace without a false replacement report. A real strategy, base-ref, mode or environment change must retain its replacement checks. **Steps to reproduce** 1. Prepare a fresh isolated remote Git retry with a `git_worktree` strategy. 2. Complete its runtime handoff. 3. Admit another turn with the same workspace policy. 4. Inspect the recorded workspace freshness decision. **Paperclip version or commit** Reproduced on `b9e94b85a69bf679818eeef9ecf4655554a89c6b`. The regression also fails on the unchanged preparation code in the current base. **Deployment mode** Authenticated server with remote workspace authority and a sandbox execution target. ## What Changed - Keep the admitted strategy when a fresh workspace uses remote authority. - Retain the temporary controller strategy for existing host-side seed preparation. - Extend the PostgreSQL preparation test through a fresh admission and its next ordinary admission. Assert the same workspace, current writer and stable fingerprint, with no false drift operation. - Cover a genuinely configured `project_primary` fresh remote Git retry followed by a real strategy change, preserving its replacement decision and stored fingerprint. - Document the execution rule and the separate live verification recipe. ## Verification - The final regression was run before and after the production change. Before: the next admission recorded a strategy replacement. After: unchanged reuse records no drift and retains the workspace and fingerprint. - All 312 targeted cases pass on `48b3e548afa42e740871feae2763d88fd5a00431` across workspace preparation, workspace/session policy and inherited session preparation. These include actual strategy/base-ref changes and the rule that explicit reuse must not hide real replacement-class drift. - All eight remote preparation cases also pass with the compatibility regression: a genuinely configured `project_primary` fresh Git retry creates a `git_worktree` row, and its later real strategy change retains the replacement decision and saved fingerprint. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Full recursive typecheck and `pnpm build` pass locally on this head. - The [full current-head CI matrix](https://github.com/paperclipai/paperclip/actions/runs/41524105868) passes: typecheck/build, all general server, chat, workspace and serialized test groups, plus Runner and E2E checks. The workflow and `run-vitest-stable.mjs` partition the complete suite. - The duplicate local `pnpm test:run` passed all 1,828 chat cases and was intentionally stopped during general-server after the same complete CI matrix passed. This is not recorded as a full local-suite pass. - Greptile and Paperclip Review both assessed this head at 5/5 with no new findings. Paperclip Review accepted the compatibility boundary after the counterexample; its check conclusion is neutral. The addressed review thread is resolved. - The regression uses real PostgreSQL and preparation policy. Runtime acquisition is controlled at its external boundary. The new fix has not been deployed; live provider resumption remains unverified for this patch. ## Risks - Existing incorrect fingerprints are not rewritten. The same stored shape can come from a genuinely configured `project_primary` remote Git retry; a Git workspace row alone cannot prove the old admission policy. Rewriting it would hide a real policy change. After preserving and inspecting retained work, an explicitly admitted fresh recovery after rollout establishes a correct new fingerprint. Ordinary reuse of the old workspace does not migrate it. Normal recovery gates remain required. - Real policy changes still use the current replacement guard. No schema, credential or authority change is included. ## Model Used OpenAI GPT-6 Codex with reasoning, repository editing, shell execution, PostgreSQL tests and browser inspection. The host does not expose the exact model variant or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:11:30 -06:00
DottaandPaperclip	482706e7dd	perf(tools): index pending profile review scopes (#18681) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - The Apps UI lists tool profiles and shows new tools that need review. > - Both services project new-tool review for each profile. > - The projection scans include rules and exact rules for each catalog row. > - This repeats millions of comparisons when many selected profiles exist. > - This pull request builds scope and selector indexes once per projection. > - The benefit is less server CPU with the same fresh review results. ## Linked Issues or Issue Description **What existing behavior does this improve?** New-tool review projection in profile listings, profile review, and app attention. **Current behavior** Each newer catalog row scans the profile's included scopes. Included catalog entries require a map lookup for each scope comparison. Rows in scope then scan all rules again to omit tools that were already reviewed. Profile listing and app attention repeat this work for every profile. **Proposed behavior** Index covered application and connection scopes once. Expand included catalog entries to their existing connection and non-null application. Index exact selectors across all rules once. Keep the ordered catalog filter and returned metadata. **Reason and benefit** A recent main-thread profile identified this scope-matching path as a large CPU consumer. The local reproduction below confirms multi-second CPU work in the actual owning services. This PR removes the repeated scans without caching authorization or changing database reads. Related [#18673](https://github.com/paperclipai/paperclip/pull/18673) indexes effective access and profile summaries. It leaves this pending-review path unchanged. This PR targets the separate pending-review mechanism and has no dependency on that PR. The selector helper is identical so the owners can share it when both changes land. ## What Changed - Build covered application and connection sets for included scopes. - Build exact selector sets for all rules, including exclusions, to omit already reviewed tools. - Preserve null application selectors, missing catalog references, review watermarks, status checks, output order, and names. - Add differential tests against the former scope/matching evaluator. - Add PostgreSQL checks through all three affected service methods, including current catalog and rule changes. - Add a reproducible PostgreSQL benchmark of complete profile-list and app-attention results. ## Verification Baseline: `1f8f2e195f4897cd1c237bdc068de95780ab0c3c`. Treatment: `fd59c0183ce64c44696f79604516f4d51ca5662b`. Node 24.22.2 on macOS arm64. The production projection file at this baseline is identical to the file at `410b0e27015495ab8951768b5b0b94727de33785`. The benchmark exercises the real `createToolProfileService(...).listProfiles` and `createToolAppAttention(...)` methods against disposable PostgreSQL. It uses 200 selected profiles, 1,900 active tools, 76,000 rules, and 12 agents. Profiles alternate between 65 and 695 rules. Half reference 695 removed catalog entries; half select current tools. This tests both the slow failed scope scans and the slow exact-rule scans. No provider calls or hosted traffic are generated. The complete ordered results from both services match baseline. After one equality/warmup invocation per version, five samples alternate implementation order. Forced GC runs before each measured pair of service calls. Other workers paused owned builds and benchmarks; unrelated host activity was not controlled. | One profile listing plus one attention listing | Baseline median (range) | Change median (range) | | --- | --- | --- | | Process CPU | 5,406.512 ms (5,221.533–5,648.262) | 1,770.116 ms (1,596.232–2,045.617) | | Elapsed time | 5,279.231 ms (4,932.419–5,537.105) | 1,532.414 ms (1,499.199–1,610.428) | | Queries | 14 | 14 | | Returned database rows | 156,215 | 156,215 | Median process CPU falls **67.3%** and median elapsed time falls **71.0%**. Profile summaries retain their old nested matching in this comparison, so the result isolates pending-review indexing. Query counts and row counts do not decrease. This is local source/tsx service proof. It does not establish hosted improvement, whole-run latency, or HTTP tail latency. Reproduce with `server/scripts/benchmark-profile-review-projection.ts`; baseline-module setup is documented in `doc/mcp-discovery-performance.md`. Passed: ```sh node node_modules/vitest/vitest.mjs run server/src/__tests__/tool-profile-pending-projection.test.ts server/src/__tests__/tool-profile-review-service.test.ts server/src/__tests__/tool-profile-entry-order.test.ts node scripts/check-module-boundaries.mjs ``` All five tests in three suites passed. The differential tests cover all selector kinds, include/exclude effects, null scope behavior, missing references, review watermarks, and exact special-character names. PostgreSQL checks preserve current pending counts after exclusions, catalog removal, and review; another company remains separate. `git diff --check` passes. The first new integration assertion used `summary` instead of the existing attention response's `totals` field. It failed before the mutation checks. The test assertion was corrected; no production code changed for that correction. The final run above passed. Server typecheck was attempted but did not pass local setup. The packaged command did not reach TypeScript because a global pnpm child tried to fetch and verify the project's pnpm version through an unavailable registry. A direct Node 24 TypeScript invocation then reported missing built `@paperclipai/paperclip-runner` and related package declarations, with cascading type errors in unchanged consumers. No error points to the changed projection. The prerequisite build was not expanded into a full workspace build. Full local unsharded tests and build were not run; final-head CI must supply broad acceptance. No merge or deployment was performed. ## Risks Scope expansion must preserve existing semantics. An individual catalog rule covers its connection and only a non-null application. An explicit null application selector still matches null applications. Removed references contribute no scope when absent from the supplied catalog. The oracle and PostgreSQL tests cover these boundaries. The indexes add linear temporary memory in rule count and live only for one projection. Current company-scoped database reads and review mutations remain in their owners. Exclusions remain explicitly reviewed; they do not create new review items. Revert this commit to roll back. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with source inspection, code execution, tests, and local PostgreSQL benchmarking. The context window size is not available in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:10:42 -06:00
DottaandPaperclip	2b374caba0	fix(recovery): preserve unfinished remote exports before Retry (#18678) ... ## Thinking Path > - Paperclip manages agent work and preserves each task's saved files. > - A completed native result can still need workspace export. > - Remote workspaces now use a different saved descriptor from legacy copyback. > - The execution hold checked only the legacy descriptor. > - Task Retry could admit another run before the original export finished. > - A later failure could then supersede the original recovery record. > - This change keeps unfinished remote exports fenced and permits explicit historical file recovery. > - Recovery keeps human review and newer execution authority intact. ## Linked Issues or Issue Description **What happened?** A native run completed successfully, but its remote workspace export failed. The recovery action used `native_workspace_sync_out_retry_exhausted`. Task Retry created a second task run while the original files still needed preservation. The newer failure then superseded the original action. A board file-recovery request could queue that cancelled record, but the worker excluded it from discovery. **Expected behavior** Keep Retry behind the unfinished export. Save the exact source and finalize the existing accepted result once. Preserve pending human review. An explicit request for a superseded source must save its files without applying an old result over newer work. **Steps to reproduce** 1. Save an accepted successful native result with a prepared `remoteWorkspace` descriptor. 2. Fail its final export and retain its exact sandbox lease. 3. Select Task Retry before preservation completes. 4. Observe another task run before the original workspace is saved. 5. Supersede the original action and request file recovery for its exact source. Observe queued work that the worker does not select. **Paperclip version** Observed at `1f8f2e195f4897cd1c237bdc068de95780ab0c3c`. The regression is present on current master before this patch. **Deployment mode** Authenticated hosted instance with native Paperclip Runner and remote sandbox workspaces. Related: #18403 introduced the remote workspace path. #18644 reduces export scratch disk use. #17916 covers a separate legacy warm-copyback policy case. This change repairs remote export admission and explicit historical preservation. ## What Changed - Include prepared remote descriptors in the existing native export hold. - Retain remote source identity in the blocker and continuation read projections. - Let the existing worker select a cancelled export-repair record only after an explicit board file-recovery request. Keep the current action and workspace binding unchanged. - Add real PostgreSQL, HTTP, filesystem and Git regressions. Update execution semantics and recovery maps. ## Verification - Red before the admission fix: HTTP Retry creates two task runs instead of one. - Red before the historical-source fix: the explicit recovery stays scheduled and saves no snapshot. - All 588 focused tests pass across the workspace recovery, recovery source, explicit continuation and native export retry suites. The three new integration cases verify duplicate request identity, original result finalization, restored bytes and Git HEAD, pending human review, current-writer exclusion, and preservation without older-result commitment over newer work. - `pnpm -r typecheck`, `pnpm build`, map validation and `git diff --check` pass. - Full hosted CI passes on `c598718379fa939ec539e8e91a15a94d90321b59`, including all server, workspace, serialized, Runner and browser lanes ([run](https://github.com/paperclipai/paperclip/actions/runs/41529739604)). Greptile reviewed that commit at 5/5 with no findings; there are no unresolved review threads. Paperclip Review also published a complete 5/5 assessment for that exact commit with zero findings. Its check conclusion is advisory/neutral, titled `5/5`. The first attempt lacked a structured assessment; one normal completion request recovered the saved analysis and published the validated assessment. - The local `pnpm test:run` passed all six chat partitions, then was stopped during the general server group after equivalent full CI completed. Its partial log is retained; it is not reported as a completed local full-suite pass. - The test provider boundary uses real temporary files and the production archive path. It simulates provider resume/stop calls. The deployed fix and original hosted task recovery remain unverified until merge and rollout. - The older local Node 24.11.0 runtime failed to load the installed tsx worker. Node 26.4.0 passed the isolated worker check. Required repository checks use an isolated Node 24.22.2 runtime. The earlier failure logs remain recorded. ## Risks - Recognizing another unfinished export can keep previously admitted work waiting. This is intentional until persistence is verified. - Explicit historical recovery can resume the exact stopped sandbox to read files. It retains source, allocation, ownership, writer and bounded-attempt checks. It starts no provider turn. - A newer run still prevents commitment of an older accepted result. This change does not infer that a cancelled run had no external effects. - No schema, credential, deployment or fleet configuration changes. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose the exact model ID or context-window size. Reasoning, repository inspection, tool use and code execution assisted this change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:03:09 -06:00
DottaandPaperclip	919ddbf46d	fix(workspaces): read committed files before live allocations (#18677) ... ## Thinking Path > - Paperclip keeps task files in managed workspaces. > - A final remote save commits a durable snapshot and clears writer authority. > - The old allocation can remain recorded while its transport becomes unavailable. > - Files still tried that live allocation before the accepted saved version. > - This PR reads the validated save when authority is clean and no writer owns it. > - Dirty work and active writers still require the live allocation. ## Linked Issues or Issue Description Related: #18403 introduced remote Files access. #18665 fixes a separate completion-settlement issue and is included in the base. **What happened?** A file preview returned `422 workspace_unavailable` after a remote task had saved its files. Its clean authority still had a live location. The same original bytes were readable from the committed save after that location cleared. The exact original live transport failure is unknown. **Expected behavior** Once the workspace is clean and has no writer, Files reads its exact current committed save. A retained location must not make accepted files depend on remote command execution. **Steps to reproduce** Create a real committed workspace archive. Keep its allocation active in the recorded authority. Clear dirty state and writer authority. Make live execution unavailable, then request the ordinary file preview route. Before this fix it returns 422. After this fix it returns the saved bytes without a provider command. ## What Changed - Select the saved-file loader before consulting a live location when authority is clean and has no writer. - Keep exact company, workspace, snapshot, archive, and authority-change checks. - Keep the live path for dirty work and active writers. Never substitute their older saved bytes. - Cover browse, resolve, preview, download, rejection controls, and saved-copy cleanup through real routes and storage. - Document the selection rule and update its feature and access-map recipes. ## Verification - Red: the new ordinary-route regression expected HTTP 200 but received 422 against the old service. - A separate compatibility control reproduced the optional-loader regression. Valid live reads still work when the loader is absent; no-location reads still fail, and a configured invalid save never falls back to live bytes. - Green: 98 ordinary file-route and authorization tests passed with real PostgreSQL and archive storage. - All 46 opt-in remote workspace lifecycle tests passed against one owned local disposable PostgreSQL database. Its database, server, and files were cleaned after the run. No live S3 or provider flags were enabled. - Full workspace typecheck passed after the optional-loader compatibility repair. Full build passed locally before that narrow repair. All normal CI checks pass on the current head, including build and server suites. The separate Paperclip Review job and the full local test suite remain in progress. - Feature/access-map validation and `git diff --check` passed. - A read-only staging check preserved and verified the original saved bytes. It did not rerun the agent or qualify the failed journey. The fixed active-location path has local regression evidence; deployed acceptance is pending. ## Risks A clean workspace now uses the existing bounded saved-archive loader while a location remains recorded. Each Files request can retrieve and extract that archive. Existing disk, size, concurrency, cancellation, and cleanup bounds remain. A dirty workspace or current writer still uses live files and fails closed when they are unavailable. This change neither releases compute nor changes access permissions. It does not identify the original provider RPC subreason. ## Model Used OpenAI GPT-6 through Codex with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:00:04 -06:00
DottaandPaperclip	5d40e75253	fix(evals): distinguish scheduled TestOps results from uploads (#18676) ... ## Thinking Path > - Paperclip lets people manage AI agents and inspect their work. > - The chat catalog reports each original Product E2E case to TestOps. > - TestOps creates scheduled result rows before a case runs. > - Those rows have identity fields but no status property. > - The publisher currently mistakes any case row for an uploaded outcome. > - This PR distinguishes scheduled membership from an outcome after strict identity checks. > - Each case can upload once and wait for processing without changing its grade. ## Linked Issues or Issue Description Refs #18647, #18666. **What happened?** A real TestOps launch had all three scheduled result rows before any model case started. Each row had the expected case, project, launch and job-run identity. Its status property was absent. The publisher treated the row as already uploaded, skipped its only POST and then rejected readback. **Expected behavior** A fully bound scheduled row permits the first immutable case upload. It does not count as an outcome. Readback waits within the existing bound until the original status and exact attachments are ready. **Steps to reproduce** Run the new scheduled-row control against the parent API. It rejects before sending the initial result. Fixing only that initial check still rejects the first scheduled readback after one POST. ## What Changed - Share the strict result-detail identity check between initial inspection and readback. - Recognize an absent status property as scheduled only after that check. Present valid statuses, including unknown, stay readback-only. Malformed statuses and foreign identities fail closed. - Keep scheduled full-catalog readback unready within the same poll and deadline bounds. Keep the final status, evidence and attachment checks unchanged. - Calibrate three planned rows, delayed and permanent processing, unknown outcomes, wrong job/source and explicit-null status. Retain the once-only upload and uncertain-response controls. Use the registered legacy full-name alias in the fixtures. ## Verification - The real-shape parent control fails with `result_mismatch` after the first upload is suppressed. - A controlled initial-only fix sends one POST and still fails on scheduled readback. The final two-seam fix passes. - All 77 focused controls pass: `node --test tests/runner-e2e/chat-catalog-testops-streaming.test.mjs tests/runner-e2e/chat-completion-testops-api.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs`. - Both changed files pass Node syntax checks. `git diff --check` passes. - All local controls use synthetic data and mocked HTTP. No local app, database, native build or provider runs. - The affected live run was cancelled during build after metadata proved no original case had started. Its build instance is terminated. No original grade was changed or inferred. - Latest-head GitHub CI is green: 48 successful jobs and one skipped. Typecheck, build, browser and required aggregate checks passed. - The first CI attempt failed a heartbeat scheduling test’s five-second idle wait after its 51-admission assertion passed. Its six direct owners were byte-identical to the prior passing CI. One same-head failed-job recovery passed the server shard and required aggregate; all 46 earlier successful jobs were carried unchanged. The original failure is preserved, and its underlying cause remains unresolved. No heartbeat source, fixture or timeout was changed. - Live streaming qualification remains pending. No user command or documentation contract changes. ## Risks - The observed JSON distinction is status-property absence, not a result-stage or UUID field. Those fields are not used. - A scheduled row must match the current project, launch, case, registered full name, job run and GitHub source URL. It never qualifies as passed or complete. - The same 12-poll and time limits remain. A permanent scheduled state fails. A present outcome never causes another POST. An uncertain write still uses GET-only reconciliation. - Backend handling of uploaded scheduled rows still needs live qualification. Duplicate, foreign and final-coverage guards remain; this PR does not assume server deduplication. - Earlier dispatch and cancelled setup records remain separate. The first integration HTTP 500 cause is still unknown. No in-flight legacy workflow is changed or rerun. ## Model Used OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does not expose a context-window size. The independent source peer uses the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 15:43:33 -06:00
DottaandPaperclip	0c48f5e727	test(runner): cover document guidance for file-only objectives (#18669) ... ## Thinking Path > - Completion guidance can link a saved task document even when the user requested only a file. > - The validator loads document publication proof when the objective requests a document or the completion report cites one. > - The success tests added in #18661 all requested both a document and a CSV. > - Those tests therefore could not detect loss of the cited-document branch. > - This test requests only a CSV and resubmits the actual guidance link with a registered file receipt. ## Linked Issues or Issue Description Addresses the [coverage warning on #18661](https://github.com/paperclipai/paperclip/pull/18661#discussion_r4447457009). **What happened?** The earlier success cases exercised the document-requested condition. The independently cited-document condition lacked regression coverage. **Expected behavior:** A file-only objective can cite a verified saved-document guidance link alongside the required registered file. **Steps to reproduce:** Publish a task document, register a CSV, request only the CSV, obtain the saved-document href from completion guidance, and submit both pieces of evidence. Removing only the cited-document lookup must make this case fail. **Paperclip version or commit:** Current master after #18661. **Deployment mode:** Local source tests with real PostgreSQL and native publication receipts; the remote CSV reader is controlled. No deployment is part of this follow-up. ## What Changed - Allow the existing registered-file test helper to accept a file-only objective. - Add one success case using real document publication, file registration, and the server-generated guidance href. - Keep production source and the existing negative cases unchanged. ## Verification - All 27 native completion feedback tests pass on current production code. - The new case fails when only `|| citesTaskDocument` is temporarily removed from publication lookup. The production file is restored exactly afterward. This demonstrates a coverage gap; it does not establish a new production defect. - Server typecheck and `git diff --check` pass. - Full repository CI passed, including both required E2E and verify aggregate gates. Greptile scored the current head 5/5 with no unresolved findings. The separate optional Paperclip review is queued and has supplied no finding. The preceding production PR records its separate local full-suite timeout and isolated passing retry; this follow-up uses the focused local suite and full CI rather than repeating that long local run. ## Risks Low risk: only the existing test file and its fixture helper change. The new case depends on real document/file publication contracts, including the CSV receipt requirement. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell execution, and automated tests. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.6	2026-11-02 15:33:46 -06:00
3 people	45e91f097c	fix(inbox): requalify existing unread fix (#18670) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The personal inbox must keep new comments unread until the person reads or answers them. > - A comment updates task recency, even when another person writes it. > - The assignee unread query treats that task timestamp as a read receipt. > - [rudycelekli](https://github.com/rudycelekli) already repaired this in #15085, including a reviewed deleted-comment safeguard. > - That PR now has merge conflicts and checks from an older base. This PR ports and requalifies the existing unread fix on current master. > - The port adds real human-comment and API regressions. The original contribution remains open and unchanged. ## Linked Issues or Issue Description Fixes: #15079 Refs: #15085 This is an attributed maintainer port of #15085 at `fe6a486c9ca21cd3a6ea0cef9764c372c53f4572`. The commit includes the verified original contributor credit. This is not an independent claim to the original repair. ## What Changed - Port the stable creation-time fallback for a human assignee in SQL and derived user context. Explicit reads and own replies still take precedence. - Port the reviewed safeguard that excludes deleted external comments. Keep the historical touch from a deleted own reply. - Exercise real comment writes, task recency updates, existing read receipts, read/unread toggles, later assignment, company isolation, and comment deletion through native PostgreSQL and ordinary API projections. ## Verification - Native PostgreSQL regressions fail before the repair: real comment writes leave the assignee falsely read, and later assignment hides existing comments. - A separate deletion regression fails before the reviewed safeguard: a deleted external comment remains the latest external activity. - The frozen head `0dadf85611cbfdc7a762cb82b03ed64519be5434` passes all 41 focused inbox, source-visibility, request, archive, and user-context tests on native PostgreSQL. No tests were skipped. - `pnpm -r typecheck` and `pnpm build` pass on that same frozen head with Node 24 and pnpm 9.15.4. The current-head CI is green, including all general and serialized test shards, build, typecheck, Runner checks, and eight E2E shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41525545915). - The local full-test attempt was stopped before the one master rebase. It is incomplete and is not a local full-suite pass. The full current-head CI completed successfully. - Fresh Greptile review is 5/5 on `0dadf85611cbfdc7a762cb82b03ed64519be5434`, with no actionable findings or open review threads. ## Risks - Assigned tasks with older external comments can appear unread. A later explicit read or own reply still clears them. - This changes no schema, API fields, company membership rules, or query joins. - Legacy compact-response cache policy is unchanged. Immediate mutation checks use the ordinary, rows, and index projections. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository inspection, code editing, and command execution. This session does not expose a more specific backend model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no documentation change is needed to restore the existing unread contract) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 15:26:28 -06:00
Huanyi Xie	3b24911c16	fix(hermes-gateway): preserve complete whitespace-sensitive delta frames (#15201) ... ## Thinking Path > - Paperclip shows agent output in a live transcript. > - Hermes Gateway emits structured events and raw message text to stdout. > - A raw chunk without a newline prefixes the next structured event. > - The anchored transcript parser then loses later message deltas. > - This pull request emits each delta once as a complete JSON event. > - The transcript can read all streamed events again. ## Linked Issues or Issue Description Fixes: #15132 Related PR #9909 removes the duplicate raw-delta log write. This PR also fixes two separate fidelity defects that its diff leaves: whitespace-only/leading/trailing delta text is trimmed by `nonEmpty`, and large delta payloads are shortened by the event-log summarizer. The final implementation preserves the exact delta in a JSON frame and the assembled output, with whitespace, 9,000-character, and forged-prefix regressions. The overlapping raw-log removal is required so the safe frame is the only transcript copy. ## What Changed - Remove the redundant raw stdout write; each delta is logged once as a JSON event. - Preserve whitespace and existing secret redaction; keep complete message deltas beyond the diagnostic truncation limit. - Exercise the persisted stdout through the real transcript parser, with consecutive chunks, embedded newlines, a forged event prefix, and a 9,000-character delta. ## Verification - The new regression finds only the first delta on the original code. - All 87 Hermes adapter tests pass after the fix. - Hermes adapter typecheck and build pass. - git diff --check passes. - All remote full-repository CI gates pass for the final head. ## Risks Low risk. Message-delta logs contain one canonical redacted delta payload. Other events keep their existing diagnostic limits. The regression proves there are no duplicate stdout entries or fabricated events. ## Model Used OpenAI Codex, based on GPT-6, assisted with source review, implementation and local test execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted local checks and full remote CI pass) - [x] I have added or updated tests where applicable (Hermes regression / real Compose interpolation checks) - [x] I have updated relevant documentation to reflect my changes (Hermes has no configuration changes; Compose documentation is updated) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-02 15:07:44 -06:00
DottaandPaperclip	b4a66027b7	fix(server): deliver chat completions after nested workspace saves (#18665) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat returns completed worker tasks to the original conversation. > - Delivery must retain its wake receipt when execution gates refuse admission. > - A workspace save can contain a nested snapshot operation. > - The old checks lost the receipt and treated that saved snapshot as unfinished work. > - This pull request preserves the receipt and recognizes a successful owning save. > - The benefit is that completed work can reach the chat through the normal queue. ## Linked Issues or Issue Description Refs #18652. Related: #17206 covers failed finalize recovery. This repair handles a successful finalize with an enclosed snapshot and does not change failed-save recovery policy. **What happened?** A delegated worker finished and saved its result. The original chat received no update. The dependency gate compared operation start times. A nested snapshot started after its enclosing finalize and therefore left a permanent hold. The completion wake also lost its exact idempotency receipt when the queue merged a refused wake into a generic dependency wait. The outbox reported `Completion wake returned no run or durable receipt`. **Expected behavior** A successful finalize covers a completed operation enclosed by that save when company, workspace, issue, and run match. Completion wakes that disable coalescing keep their own durable receipt even when admission waits. Dependency, budget, Stop, and pause gates still apply. **Steps to reproduce** 1. Hand a task from a chat to a worker and make the chat wait on that task. 2. Finish a workspace finalize that starts before its nested snapshot and ends after the successful snapshot. 3. Mark the worker Done and deliver its completion outbox entry. 4. On the parent revision, the dependency stays blocked and the completion receipt is absent. **Paperclip version or commit** Reproduced against `53f480d1b3` with real disposable Postgres rows and the normal queue/outbox services. ## What Changed - Honor `allowRunCoalescing: false` when recording execution waits. - Accept an enclosed successful operation only when the successful finalize has the same company, workspace, issue, and run and covers its whole recorded interval. - Preserve the existing rule that a strictly later successful workspace finalize clears earlier work. - Add failure controls for foreign runs, unfinished snapshots, failed saves, snapshots that finish too late, and later operations. - Exercise the real outbox and queue together. Hold provider dispatch at the existing drain boundary and verify server-recorded Done facts reach the admitted turn. - Document receipt identity and workspace-save coverage. ## Verification - Before repair: the two causal regressions fail and five safety controls pass. The missing exact receipt and empty wakeable-dependent list match the observed failure. - After repair: those seven cases pass without assertion changes. - The full affected set passed 288 tests and failed one existing scheduling test. Its unchanged Node24 baseline also failed. A closed-phase probe showed the fake adapter finished successfully at 3.453 seconds, after the incidental 3-second poll. The test now joins the existing execution promise and retains its status and dependency assertions under the unchanged Vitest timeout. The synchronized full dependency suite passes all 13 cases. - `pnpm -r typecheck` and `pnpm build` pass on pinned Node24/pnpm9. `pnpm test:run` is running. The combined real outbox/queue case passes. Fresh Greptile is 5/5 on `d7cee24b7032c7fdf171bfd1246cd90d41db1cba`. Exact-head normal CI and required `ci / verify` and `ci / e2e` pass. - The original scheduling failures on Node26 and Node24, the unchanged-source baseline failure, and the closed-phase probe are retained in the validation record. No model, deadline, production assertion, or paid attempt was retried to change the failure grade. - Current-head CI attempt1 passed all product and workspace coverage but failed the unchanged serialized retry-worker case at `issue-scheduled-retry-routes.test.ts:606` (first batch not observed before the poll deadline;410 other shard tests passed). The exact unchanged case passes locally on Node24 with real Postgres. The one failed-jobs-only retry passed on the same head; the original grade is retained. - The full local command completed all1828 chat integration tests. Its general stage remains running and has retained failures in unchanged `workspace-runtime.test.ts` (30-second local port-conflict fixture) and `muse-control.test.ts` (current binding authentication fixture). These are not counted as passing local full-suite evidence. Their current-head hosted coverage passed. - The separate advisory Paperclip Review app is paused and has no code finding. Its in-progress state is preserved and is not counted as a pass; it is not a required repository status check. - The live Cloud completion journey remains failed and unqualified. It must pass at the deployed fixed revision before daily-plan admission. No QA oracle, model, budget, or run limit changes are included. ## Risks - The workspace coverage query must not clear a save from another owner or new work after finalization. The guard checks the full interval and exact owner identity, with negative controls. - Refused noncoalescible automatic wakes retain separate receipts. Replaceable automatic signals still merge. No new polling or provider retry path is added. - No migration or API change. Deployment and live acceptance are separate remaining gates. ## Model Used - OpenAI Codex, GPT-6, with reasoning, tool use, and code execution. The exact deployment variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 15:06:26 -06:00
DottaandPaperclip	cd12432947	fix(runner): recognize verified task document evidence (#18661) ... ## Thinking Path > - Paperclip manages AI agents and keeps their results attached to tasks. > - Native tools return links to saved task documents. > - Completion feedback asks agents to include those links. > - The file-delivery validator treated every such relative link as a workspace path. > - A valid document and CSV result could therefore fail completion when it cited the supplied document link. > - This change derives document links from the same verified publication projection for guidance and validation. > - Requested files still need their own verified delivery evidence. ## Linked Issues or Issue Description **What happened?** The native document tool and completion feedback produce a link such as `/EX/issues/EX-1#document-output`. When a task also requests a downloadable CSV, citing that link causes completion to reject it as a workspace-only file. This occurs even when the document is published and the file has a valid current-run delivery receipt. **Expected behavior** Accept a server-derived document link only when it names the current company/task and a currently attached revision with a matching native publication receipt. Keep the separate requested-file requirement and all foreign, stale, missing, and workspace-file guards. **Steps to reproduce** 1. Save a task document through the native `write_document` tool. 2. Publish a CSV through `register_deliverable`. 3. Ask completion feedback for the final-response guidance. 4. Cite its returned document link alongside the file receipt in the next completion report. 5. The old validator rejects the valid document link as a workspace-only file. **Paperclip version or commit** Current source checkout. The regression exercises the real native document producer and completion validator. **Deployment mode** Local source verification on macOS arm64 with Node 24.22.1, pinned pnpm 9.15.4, locked dependencies, temporary PostgreSQL, and a controlled remote file-reader boundary. No live provider or hosted deployment is included. ## What Changed - Derive the canonical document href in the existing company/task/current-revision publication query. - Use that same verified href in completion guidance and reference validation. - Keep file registration and hash/size receipt requirements separate from document evidence. - Cover evidence, artifact, and criterion references; foreign companies/tasks; bogus keys; local paths; and lost current-revision publication proof. - Exercise actual native document writes and CSV registration, including attachment download query and fragment links. - Update the execution contract and the document feature/access maps. ## Verification - Red before the production edit: native completion generated a real saved-document link, then rejected the same link alongside a valid published file. The failure was the workspace-only-file guard. - Green: 210 tests across native completion feedback, explicit output validation, and the existing native file-handoff suite pass. These use real PostgreSQL and native tool receipts; the remote file reader is controlled. - A saved document link alone still fails the explicit CSV requirement. A document whose current revision lacks its publication receipt is rejected. Foreign task/company links, missing keys, malformed links, and workspace paths remain rejected. - Full workspace typecheck, server typecheck, and production build pass. - `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check` pass. Map checks establish structure only. - Full `pnpm test:run` is running separately. There is no full-suite verdict yet. CI and review are pending. - Hosted acceptance and any recovery of existing tasks remain separate steps. This PR does not change final-response recovery or scheduling. ## Risks - An overly broad URL exception could admit foreign or unpublished output. The validator accepts only exact hrefs from the authenticated publication projection. - Document evidence could hide a missing requested file. A separate regression keeps that delivery gate effective. - The shared projection adds current task/company joins. Existing document feedback and publication tests cover those consumers. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell execution, and automated tests. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 15:02:00 -06:00
DottaandPaperclip	aa6c9c3f25	perf(storage): overlap streamed S3 multipart uploads (#18659) ... ## Thinking Path > - Paperclip manages AI agents and preserves their work. > - Large workspace and skill objects use the shared S3 multipart uploader. > - The uploader waits for each part acknowledgement before it sends the next part. > - Storage request latency therefore accumulates once per part. > - This change overlaps up to four streamed part requests. > - One source reader preserves byte order and backpressure. > - Failure cancels and settles active requests before cleanup. ## Linked Issues or Issue Description Related: #18082 added the shared streamed multipart uploader. #18644 fixes temporary disk use during workspace capture. It changes a separate stage. **What happened?** The shared S3 multipart uploader sends large object parts in sequence. The code waits for each `UploadPartCommand` before it consumes the next part. A controlled HTTP test confirms that only one complete part reaches the endpoint while its acknowledgement is held. **Expected behavior** Upload independent parts concurrently within a fixed limit. Preserve exact bytes, ordered completion receipts, source backpressure, cancellation, and full snapshot verification. **Steps to reproduce** 1. Upload a large stream through the S3 multipart provider. 2. Hold each successful part acknowledgement at a disposable storage endpoint. 3. Observe that the old uploader sends only one complete body until its acknowledgement returns. 4. Repeat with this change. Four complete bodies can reach the endpoint before the first acknowledgement. **Paperclip version or commit** Current source checkout. The affected uploader was introduced in #18082. **Deployment mode** Local source verification on macOS arm64 with Node 24.22.1, pinned pnpm 9.15.4, and locked dependencies. The HTTP tests use synthetic data and a disposable loopback endpoint. ## What Changed - Send up to four part requests at once through bounded streams. - Keep one source reader and retain the existing adaptive part-size policy. - Store successful receipts by part number before completion. - Cancel a stalled asynchronous source read without waiting for the source to produce more bytes. - Preserve the first error. Cancel sibling requests and settle them before multipart abort. - Cover exact bytes, out-of-order acknowledgements, backpressure, the concurrency limit, source errors, cancellation, and the real S3 HTTP client. - Document the transfer bound. Snapshot encryption, publication, and full object verification retain their existing behavior. ## Verification - Failing before the change: the held-acknowledgement regression expected four complete part bodies and received one. The same regression now passes. - All 15 focused multipart and storage-diagnostic tests pass. The real AWS S3 SDK sends four exact 8 MiB bodies through a disposable HTTP endpoint before any acknowledgement. Completion retains source order. - Controlled transfer comparison: one synthetic 64 MiB stream, eight 8 MiB parts, real S3 SDK and loopback HTTP, each acknowledgement delayed 100 ms. Original: 904 ms, peak one request. Fixed: 247 ms, peak four. Both transfers verified every part hash and completion order. This is local boundary evidence, not a hosted latency claim. - A second safety regression reproduced cancellation waiting indefinitely on a stalled asynchronous source. Abort-aware reads now settle within the existing bounded cleanup budget. - All nine existing encrypted snapshot storage tests pass after correcting missing library aliases in the local embedded-Postgres dependency. The failed setup attempt had no test verdict. - Full workspace typecheck and production build pass. The rebased server typecheck also passes. - The first full `pnpm test:run` attempt stopped during chat test collection with `No test found ... in lines ...` before a suite verdict. The chat test and line-selection scripts are unchanged from master. A retry after local Postgres dependency setup passed its first chat shards, then was stopped to avoid redundant host load after current-head CI passed every type/build/general/serialized/runner group. The unrelated narrow-banner E2E job failed its first attempt and passed one bounded retry; all current-head GitHub CI jobs are now green. There is no full local verdict. The separate Paperclip native review concluded neutral after reporting that work was paused; it supplied no substantive code review. - Commands: `pnpm -r typecheck`, `pnpm build`, and `pnpm --filter @paperclipai/server exec vitest run --configLoader runner src/__tests__/storage-s3-multipart.test.ts src/__tests__/storage-s3-failure-diagnostics.test.ts src/services/remote-workspace-snapshot-storage.test.ts`. - Deployment and an end-to-end hosted acceptance run remain separate steps. ## Risks - Four requests per upload increase concurrent storage traffic. Multiple simultaneous uploads multiply that bound. - Shared source consumption could reorder bytes. One producer and exact-content tests protect that boundary. - Failed siblings could outlive multipart cleanup. Cancellation and settlement tests protect cleanup order. - Capture and full read-back verification still take time. This change only targets serial multipart latency. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, code editing, shell execution, and automated tests. The context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 15:01:34 -06:00
DottaandPaperclip	410b0e2701	fix(evals): accept refreshed catalog workflow URL (#18666) ... ## Thinking Path > - Paperclip lets people manage AI agents and inspect their work. > - The chat catalog reports original Product E2E grades to TestOps. > - TestOps can refresh its job metadata from GitHub. > - That operation returns a canonical master workflow-file URL. > - The catalog guard currently rejects that exact URL before case selection completes. > - This PR accepts that fixed URL and keeps the existing source and identity checks. > - The catalog can use the refreshed job without changing its cases or grades. ## Linked Issues or Issue Description Refs #18542, #18647. **What happened?** A supported TestOps job refresh changed the catalog job URL to the GitHub `blob/master/.github/workflows/chat-catalog-testops.yml` form. The selection guard rejects it as `job_mismatch`. **Expected behavior** The guard accepts that exact parent workflow URL. It still rejects another owner, branch or workflow. **Steps to reproduce** Run the new adapter regression against the parent commit. It fails in `validatePilotSelection` before the job-run request. ## What Changed - Add one exact canonical master URL to the full-catalog workflow URL guard. - Add a regression through the real selection validator with refreshed job metadata. Keep negative checks for another owner, branch and the called case workflow. ## Verification - The new selection regression fails on the parent with `job_mismatch` and passes with this change. - All 73 focused controls pass: `node --test tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-completion-testops-api.test.mjs tests/runner-e2e/chat-catalog-testops-streaming.test.mjs`. - Both changed files pass Node syntax checks. `git diff --check` passes. - These controls use synthetic metadata and mocked HTTP. No local app, database, native build or provider runs. - Removing the one added predicate restores the parent API file exactly. The branch, job, case, source, evidence and grade checks remain unchanged. The pilot URL policy also stays unchanged. - Current-head GitHub CI passes: 48 jobs pass and one is skipped, including the required typecheck, test, build and final summaries. Live streaming qualification remains pending. No documentation change is needed for this internal URL compatibility fix. ## Risks - The alias accepts only this repository, master branch and parent workflow in the full-catalog lane. It grants no alternate source or execution authority. - An earlier TestOps dispatch failed with an integration HTTP 500 before GitHub created a run. Its cause remains unknown. This patch addresses the separate, reproduced metadata compatibility failure; it does not claim to cause or cure that HTTP 500. - Existing in-flight workflows keep their original source. This PR does not patch, rerun or upload into them. ## Model Used OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does not expose a context-window size. The independent source peer uses the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 14:53:30 -06:00
Devin FoleyandPaperclip	49b3804830	fix(connections): load permissions and show AI account identity (#18653) ... ## Thinking Path > - Paperclip manages AI agents and their access to saved connections. > - The Permissions page loads account details and permitted agent choices. > - Company-wide AI accounts repeat retained installation checks for every candidate agent. > - This makes target checks grow with the square of the roster size. > - The browser waits without a deadline, and connection names do not identify the provider login. > - This pull request batches direct AI checks and bounds loading with recovery controls. > - Account details load separately and show a provider email or an explicit profile-permission limit. ## Linked Issues or Issue Description **What happened?** Opening a saved AI account can leave Permissions on “Loading AI account…” for minutes. The account itself can be connected. The existing capability batch excludes AI connections, so a company-wide install repeats checks for every installed agent for every candidate. Saved names and Paperclip ownership also do not establish which provider login a credential uses. **Expected behavior** Direct AI accounts reuse retained-target checks within one descriptive request. The page shows loaded account controls or a bounded failure with Retry. Permissions shows the provider email and organization when the selected credential permits profile access. An unavailable profile must not block maintenance or disconnect a working inference credential. Writes still authorize current permissions. **Steps to reproduce** 1. Save an AI account in a company with several agents. 2. Install the account company-wide. 3. Open its Permissions page. 4. Delay the account-list or grants response to exercise the loading failure. The maintained benchmark reproduces database work with synthetic local data. Refs #17945, which added batching for non-AI connections. Searches found no open PR for this AI-account loading fix. ## What Changed - Include direct AI accounts in the existing board capability batch. Referenced pool members, delegated actors, complex selectors, and mutation authorization keep their existing evaluator. - Add opt-in per-caller GET deadlines to the API client. A timed-out caller detaches the stuck coalesced request so Retry starts a fresh fetch. Other callers retain their own cancellation and results. - Apply a 15-second deadline to AI account lists and grants reads. The permissions view retries transient failures once and reports other failures immediately. - Share the parent Permissions grants query with AI account details. Keep loaded controls visible after a background error and offer Retry. - Add a grant-scoped identity GET and OpenAPI contract. The service checks current membership and credential audience before it reads the selected credential. It projects only the provider email and organization; provider IDs, tokens, and raw errors stay private. - Show account details independently of maintenance controls. Anthropic profile denial has an explicit message and preserves connected status. Transient profile failures have a separate Retry action. Observations are cached for five minutes under the company, connection, grant, and grant revision. - Extend real PostgreSQL and authenticated HTTP coverage, client/component regressions, map guidance, and the existing benchmark's AI mode. - Address the Apex file-size finding by moving gallery identity lookup into the existing app display helper module. `AppDetail.tsx` is 984 lines; lookup behavior is unchanged. ## Verification Local source checkout on macOS, Node 25.6.1, pnpm 9.15.4, and disposable PostgreSQL 18.1. Provider calls use fixtures. No hosted deployment or provider qualification is claimed. - Passed `pnpm -r typecheck` and `pnpm build`. - Passed all 87 tests in `connection-management.integration.test.ts` and all 33 tests in `connection-install-capabilities.test.ts`. - Passed 196 AI connection, identity-provider, and OpenAPI tests: `ai-connections.test.ts`, `ai-account-identity.test.ts`, and `openapi-routes.test.ts`. - Passed 107 focused UI/client tests: `client.test.ts`, `ManagedAiConnectionDetails.test.tsx`, `AiConnectionAccountDetails.test.tsx`, and `AppDetail.test.tsx`. - After the Apex extraction, passed UI typecheck/build and 106 component tests across `AppDetail.test.tsx`, `AppNotConnected.test.tsx`, `ManagedAiConnectionDetails.test.tsx`, and `AiConnectionAccountDetails.test.tsx`. - The authenticated HTTP tests compare complete grants responses across human roles, for tool and AI connections. They verify a real install mutation succeeds before revocation and fails afterward. - A company-wide personal AI fixture checks 20 targets instead of 400. Pool-member permission loss still denies the complete pool description. - UI states covered: shared cached grants, stuck account/grants fetches, one automatic retry, terminal failure, manual recovery, authorization failure, background failure with retained controls, and unmount cancellation. Client tests also cover stalled response bodies and independent coalesced callers. - Identity coverage includes actual bearer-key/JWT reads, private-user and cross-company denial before provider access, revoked credentials, provider 401/403/429/500, and grant-change isolation. A real local HTTP fixture stalls the response body, verifies the shared 15-second deadline, and keeps unrelated HTTP requests responsive. The test shortens the native deadline to 100 ms. Profile permission denial leaves the saved account connected. - Passed `pnpm check:token-gates`, `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The full local `pnpm test:run` started with the loading-only change. All six chat shards passed. It recorded one failure in the unchanged `heartbeat-retry-scheduling.test.ts` quota-rejection case. That exact case passed a focused rerun on the final head. The initial failure cause remains unconfirmed and its local log is retained; this is not a full local-suite pass. The redundant local run was stopped with SIGINT after final-head CI completed successfully. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/41515556725) passed on `ead8d276d4a2023614fff54b3e9a858f28c3178b`: typecheck, build, all 12 server shards, all 3 chat shards, workspace suites, all 9 serialized suites, all 8 end-to-end shards, Runner checks, and the aggregate verify gate. Security checks passed. Optional Storybook checks were skipped by workflow policy. GitHub reports the branch mergeable without conflicts. - Apex reviewed head `ead8d276d4a2023614fff54b3e9a858f28c3178b` at 5/5, confirmed the extraction resolves its earlier file-size finding, and has no unresolved review threads. The existing benchmark ran for the loading-only change now recorded in commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce`, with `--purpose=ai --baseline-ref=026fd9e3452`. The identity follow-up leaves that batch source unchanged. Two warmups and five alternating samples exercise each of eight fixture shapes. Complete positional capabilities, reasons, and operations match the baseline in every sample. | AI fixture | SQL calls before → after | Target reads before → after | | --- | ---: | ---: | | 1 agent, company install | 9 → 9 | 1 → 1 | | 32 agents, company install | 5,248 → 164 | 1,024 → 32 | | 100 agents, company install | 50,400 → 504 | 10,000 → 100 | | 100 agents, partial install | 13,175 → 503 | 2,575 → 100 | These counts measure the real service over local PostgreSQL. They do not predict hosted page latency. To reproduce the standalone benchmark, use a checkout of commit `5f725b36fa52002545241a5a8b7a10602f0cc0ce` and run the command below. Its isolation check intentionally rejects the later identity follow-up's additional runtime changes. ```sh node --import ./server/node_modules/tsx/dist/esm/index.mjs scripts/benchmark-connection-install-capabilities.ts --purpose=ai --baseline-ref=026fd9e3452 ``` Run the final-head regression suites from the repository root: ```sh cd server pnpm exec vitest run src/__tests__/connection-management.integration.test.ts src/__tests__/connection-install-capabilities.test.ts src/__tests__/ai-connections.test.ts src/services/ai-account-identity.test.ts src/__tests__/openapi-routes.test.ts cd ../ui pnpm exec vitest run src/api/client.test.ts src/components/ai-connections/ManagedAiConnectionDetails.test.tsx src/components/ai-connections/AiConnectionAccountDetails.test.tsx src/pages/apps/AppDetail.test.tsx ``` ## Risks Descriptive permissions reuse retained facts within one request. A concurrent change can leave a displayed choice stale; the next request and every write still authorize fresh. Pools with referenced members retain their existing cost. The account and grants clients now bound each caller's wait. A browser abort does not cancel an authorization read already admitted by the server. The independent profile request uses a fixed Anthropic origin, a bounded body, no redirects, and a 15-second deadline. Inference-only credentials can return no email; that is reported without guessing from a Paperclip owner or changing connection health. Profiles are fetched only for the caller's authorized selected grant. No ambient CLI identity, persistent identity cache, background polling, or billing-identity change is added. Other API reads keep their existing behavior. No migration is required. Staging has not been redeployed. ## Model Used OpenAI `gpt-6.1-sol` through the Codex harness in T3 Code, with xhigh reasoning, code editing, execution, and test tools. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 12:29:03 -08:00
DottaandPaperclip	33fc7fcddc	fix(db): clean up owned PostgreSQL test servers (#18627) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Database, server, and CLI tests use real PostgreSQL fixtures. > - Each fixture must own its server until PostgreSQL has stopped. > - The old teardown returned after five seconds even when its server remained alive. > - A startup failure could then mark the host unsupported and skip database coverage. > - This pull request waits for owned shutdown and makes supported-host startup failures fail collection. > - A shared-server command reduces process creation while keeping each fixture's database isolated. ## Linked Issues or Issue Description Refs: #7756 for related development-process ownership. That development-server issue remains outside this test-fixture fix. Related PR: #16799 collects macOS startup diagnostics. This PR changes test shutdown and failure behavior. **What happened?** Repeated test runs can exhaust macOS PostgreSQL semaphore resources. The fixture abandoned slow shutdown and swallowed stop errors. The support probe converted ordinary startup failures into skipped integration coverage. **Expected behavior** Tests must stop their PostgreSQL servers, retain files if stop fails, and report resource exhaustion as a failure. Fixtures on a shared server must not close another database's clients. **Steps to reproduce** Start a fixture whose PostgreSQL stop exceeds five seconds, or make a supported-host startup fail with semaphore exhaustion. The former helper returned from teardown or reported unsupported coverage. The new regression tests exercise both paths without exhausting host resources. **Paperclip version or commit** Source checkout based on current master. **Deployment mode** Local source development and CI tests. ## What Changed - Add explicit ownership for test PostgreSQL children. Wait for fast shutdown, then request immediate PostgreSQL shutdown if needed. Keep cleanup idempotent and fail when exit cannot be confirmed. - Publish reusable templates only after clean shutdown. Do not publish a crashed or immediately stopped cluster. - Stop owned children on POSIX worker signals and process exit. Preserve test directories when shutdown fails. Remove conflicting dependency exit hooks. - Retry port collisions only. Fail collection for resource exhaustion, unavailable configured servers, and other startup failures on supported platforms. Preserve skips for explicitly unsupported platforms. - Move administrative-server support into the common fixture used by database, server, and CLI tests. Close clients at database scope and migrate cloned templates before use. - Add a directly invoked shared-server runner to own one test server and run suites against separate disposable databases. Preserve child exit status and support an existing disposable server. - Unify cancellation: stop and wait for the test process group before PostgreSQL teardown. Add nested-process coverage, real worker-exit coverage, shutdown regressions, shared-database isolation checks, and setup documentation. Replace ignored heartbeat completion polls with the existing execution drain before asserting persisted outcomes. - Finish owned cluster-directory cleanup after signal-driven cancellation. Preserve prepared templates across shared-fixture tests. Remove stopped directories asynchronously, using native removal on POSIX with a Node fallback. - Align database and CLI fixture hook budgets with the existing 30-second server budget. This lets the ten-second shutdown protocol finish and report errors. The idle-sleep suite uses the existing 90-second fixture budget for its unusually large final directory cleanup. ## Verification - The semaphore-exhaustion regression failed against the old helper because it resolved to `supported: false` instead of rejecting. - All 37 focused tests pass on the final runtime code (`e4de00ab40`). They include real worker exit, SIGTERM, nested-process SIGINT/SIGTERM cancellation with PostgreSQL available during child cleanup, failed template shutdown, unsupported platforms, stale SQL templates, and directory removal. A child exiting with status 7 makes the shared-server runner return status 7 after cleanup. - Full local `pnpm -r typecheck` and `pnpm build` pass on the final runtime code (`e4de00ab40`). Subsequent changes only adjust server-test lifecycle waits, the idle-sleep cleanup budget, and documentation; the server typecheck also passes with those final tests. - A fresh shared-server run on `90b65fd8ad` passed all 68 database files and 297 tests. Earlier preparation runs had two ten-second teardown timeouts and a stale-template failure; the final helper migrates cloned templates and permits the full shutdown budget. These earlier failures remain accounted for here. - The local full repository attempt stopped on a Discord receiver-lease test after 338 passing chat tests. That exact case passed in isolation. This full-suite attempt is not a pass. - On `90b65fd8ad`, [server shard 6](https://github.com/paperclipai/paperclip/actions/runs/41496495253/job/125452949369) and its one unchanged retry reported two 15-second cached-input heartbeat timeouts and a 30-second idle-sleep teardown timeout. All ten cached-input cases then passed locally with their original deadlines. A local parallel run of the full files reported eight behavioral/hook failures and two suite teardown timeouts (281 tests passed). - Serial idle-sleep runs passed all 225 test assertions but overran the 30-second final hook. Tracing measured client closure in 4 ms, PostgreSQL exit after 5 seconds, and synchronous directory removal taking another 35 seconds. Asynchronous Node removal and native removal also exceeded that hook under the full workload. Cleanup now remains asynchronous and awaited, and only this heavy suite's final hook uses the existing 90-second fixture budget. No PostgreSQL shutdown deadline or test assertion was weakened. The serial run then passed the complete idle-sleep file, including all 225 tests and teardown. Its heartbeat file passed 63/64 cases, exposing an ignored three-second completion poll. After replacing that poll with an execution drain, another timer case exposed the same pattern (63/64 passed). All remaining ignored completion polls now await execution drain without changing test deadlines. The complete heartbeat file then passed all 64 tests. The intentional asserted lock-contention poll is unchanged. The final server typecheck passes, and a process audit found no PostgreSQL server left by the completed local runs. - On final head `2cf316d3f571e4516890cba743ecffeb9703cce3`, the previously failing server shard 6, database lane, full typecheck, and build passed. The [first CI attempt](https://github.com/paperclipai/paperclip/actions/runs/41506358136) then lost self-hosted runners in the canary and serialized-server lanes, with additional jobs cancelled. The server-2 lane had one 30-second timeout in the unchanged workspace-recovery HTTP Retry test (2,282 other tests passed). That exact test passed locally with its original deadline (28.78 seconds including imports). One unchanged retry of only failed and cancelled jobs completed successfully on the exact final commit (workflow attempt 2). The workspace-recovery timeout case and every interrupted lane passed; successful first-attempt jobs were retained. The additional Paperclip Review check remains queued, waiting for its assigned agent; it has no unresolved GitHub review threads. Greptile completed review on this exact final head at 5/5, with zero unresolved review threads. All four original review findings and their threads are resolved; additional Windows shutdown, cancellation-directory, and template-reuse findings are fixed. - `pnpm check:module-boundaries` and `git diff --check` passed. Post-run process checks found no PostgreSQL servers left by the completed verification runs. ## Risks - Immediate PostgreSQL shutdown can require crash recovery. This fallback applies to disposable test clusters and runs only after fast shutdown times out. - No process can handle `SIGKILL` or machine power loss. Host resource cleanup remains a separate backstop. - Directory cleanup on cancellation applies to the shared-server runner's own cluster. Directly terminated test workers stop their PostgreSQL process but retain disposable files for inspection. The process-exit guard cannot await file removal. - The two-second `pg_ctl -W` deadline bounds signal dispatch. It is separate from the five-second and thirty-second server-exit waits. A control-command failure fails cleanup and retains its directory; it does not report successful cleanup or force-delete live files. - Shared fixtures require a disposable PostgreSQL server and a role that can create databases. The directly invoked shared command is opt-in; the existing `pnpm test` command remains available. - These changes do not stop unrelated development servers or delete host-wide IPC resources. Clean shutdown uses the owned `pg_ctl` binary on every supported platform. Its real control path was tested on macOS. A Windows host was not available for live verification; POSIX signal tests skip on Windows. ## Model Used - OpenAI Codex, based on GPT-6. The runtime does not expose the exact serving variant or context-window size. The agent used reasoning, shell execution, code editing, and GitHub tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 14:24:48 -06:00
DottaandPaperclip	53f480d1b3	fix(native): preserve warm continuity and deferred review replies (#18631) ... ## Thinking Path > - Paperclip manages AI agents and their saved work. > - Product E2E tests must check saved personal files across separate turns. > - The server records file collection in the durable `instruction_save` run-log event. > - The warm test read optional terminal result metadata instead and rejected a recorded successful save. > - This pull request reads the complete event stream for the current run and uses its latest save receipt. > - The prompt now specifies the exact nine bytes required by the temporary-file transfer count. > - Human review actions select the actual interaction card, which can share its interaction ID with a navigation button. > - A later real three-turn run exposed a missing reply after workspace finalization moved to its durable recovery owner. > - Reply recovery handled completed tasks but omitted ordinary tasks that awaited review. > - This pull request also publishes their recorded final reply once while preserving the pending review. ## Linked Issues or Issue Description **What happened?** The [retained live continuity attempt](https://github.com/paperclipai/paperclip/actions/runs/41477302876/job/125420001074) saved the correct memory file and unchanged 8 MiB binary. Its durable run log contained a successful `instruction_save` receipt. The oracle read `resultJson.instructionSave`, which was absent, and failed before turn two. The temporary file also had a trailing LF. Its prompt allowed that result, while the byte-count check required nine bytes. **Expected behavior** The oracle must use the documented authoritative save event. A missing receipt, a later failed save, incorrect transfer counts, or another run's events must fail. The prompt must specify the bytes that its independent check requires. **Steps to reproduce** 1. Run `daytona-warm-continuity.runner-codex.daytona.warm-three-turn` with the configured protected credentials and immutable image. 2. Complete a native turn whose terminal result has no `instructionSave` field. 3. Compare the oracle output with the same run's successful `instruction_save` event. A subsequent [three-turn attempt](https://github.com/paperclipai/paperclip/actions/runs/41495337334/job/125451304553) passed every personal-file assertion and reused the same warm process. Its first final reply was recorded as a completed agent-message event but was absent from comments. The log shows `native_workspace_finalization_busy`; the durable recovery path excluded `needs_review` replies. The final message matcher correctly failed. This failure remains recorded. The observed application revision was `aab87ce70744660a4a21c77278b0ede267c04720`. Related work: #14695 and #18494. The retained failed result keeps its original grade and definition. ## What Changed - Read all run-event pages through the existing bounded collector. - Require events to belong to the requested run and select the latest `instruction_save` event. - Retain strict byte counts and reject missing, failed, pending or superseded receipts. - Specify nine ASCII bytes and no trailing newline for the temporary file. - Scope human review actions to the interaction card and its exact interaction ID. - Advance the warm suite definition to version 5 and document its evidence source. - Recover recorded final replies for ordinary native tasks with an accepted `needs_review` result and terminal state. - Preserve pending reviews, existing reply selection, company/turn/digest checks and external governed-response authorization. - Update the task feature and agent access maps. ## Verification - Real PostgreSQL regression: deferred review reply discovery failed before the production repair (expected one candidate; found zero). The final-message event and pending review were present. - Recovery cases cover one publication, no repeated provider run, unchanged review, and rejection of absent/wrong-turn final messages, bad digest and suppressed comments. - Red: seven new boundary regressions failed against the old receipt lookup. - Green: 169 focused oracle, event pagination and catalog checks passed. Product E2E typecheck passed after rebuilding local package declarations. - Retained-evidence replay confirms that the new lookup finds the real successful save event. The original attempt remains failed because its temporary file included an LF. - Recursive typecheck passed before the final recovery change; server typecheck passed afterward. The recovery and control-plane suites passed all 112 tests. Full build and map checks passed. - The broad local test run reached one unrelated 30-second timeout in `attention-retention-work.test.ts` (maintained-review/confirmation batching). The same unchanged case then passed on one targeted retry (39.9 seconds including module load). The original failure remains retained. The broad run also reported one unchanged workspace-conflict case, which passed on its targeted retry (14.85 seconds including module load). Both original failures remain retained. The redundant broad local run was stopped after current-head CI completed all required gates: 55 successful checks and 3 selection skips. No all-green local broad-run claim is made. - The separate artifact-transfer cell already passed on the recorded application revision: independent source and delivered bytes matched at 407,485 bytes, with a completed task, succeeded native run and cleared execution locks. This does not establish a three-turn continuity pass. - [Real native three-turn acceptance](https://github.com/paperclipai/paperclip/actions/runs/41499102994/job/125465795210) **passed** at `3b207a42943e9eeb371d4f2d5623de72093eb6d5`, suite definition 5. Turn one exercised `committed_task_response_recovered`. All three completed final replies were persisted once and in order. All three strict personal-file receipts passed. The unchanged 8 MiB binary survived; later transfers copied only 36 and 54 bytes. The lease was created, resumed, and resumed with the same runner PID/start time, runner instance, native session and provider session. Final task was `done`, all three runs succeeded, and checkout/execution locks were null. - The aggregate startup summary has no separate provider-process PID proof. The raw run records independently prove the stable runner process and provider session stated above. This single native cell does not qualify the legacy cell or a full provider matrix. - The passing campaign reused the signed immutable runner image. It required no disk expansion. Earlier campaign image builds used the recorded temporary 120 GiB mitigation; default 60 GiB build capacity remains unqualified. All scoped completed probes were removed and independently returned 404; org CPU/disk usage returned to zero. This does not resolve unknown historical resources under a different credential identity. - Current-head Greptile is 5/5; review-thread pagination is complete with zero threads. All 55 current-head checks are successful; 3 selection checks are skipped. The paid campaign, result enforcement and report publication all completed successfully. ## Risks - Publishing a reply must not approve the work. Recovery only copies the accepted turn’s completed final message. It preserves the pending review and does not start an executor. - Conversation and external governed-response paths retain their existing separate authorization. The latest save event can record failure after an earlier success. The oracle rejects that case. Event pagination, missing receipts and cross-run evidence also fail closed. The exact temporary-file byte count remains required. The review selector still requires the intended interaction ID and enabled action on its actual card. The production change is limited to ordinary internal-task reply recovery. It still requires an accepted result, a matching successful terminal disposition, the completed final message, and existing publication authority. ## Model Used OpenAI Codex, based on GPT-6, with shell and code tools. The exact deployment variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.5	2026-11-02 14:19:48 -06:00
DottaandPaperclip	0fd82e386e	feat(evals): stream completed chat catalog results to TestOps (#18647) ... ## Thinking Path > - Paperclip lets people manage AI agents and inspect their work. > - The chat catalog uses real Product E2E cases and reports their original grades to TestOps. > - The current report matrix waits for every selected worker. > - A large selection can hide completed results for hours. > - Each case can use the existing private reader and safe exporter after its own worker ends. > - This PR gives each case a separate upload and readback job, then verifies final coverage without another upload. > - People can inspect completed cases while the rest of the selection runs. ## Linked Issues or Issue Description Refs #18542, #18625. **What existing behavior does this improve?** The existing chat catalog reports results to TestOps only after all selected workers finish. **Current behavior** One slow case holds back the report and upload jobs for all other cases. **Proposed behavior** Each finished case gets its own protected reader, safe projection, upload and backend verification. The final job checks the whole selection through read-only requests. It does not upload results again. **Reason and benefit** A person can inspect a completed grade and its safe evidence receipts while other cases run. This changes future runs. It does not change an in-flight run. ## What Changed - Add one same-commit reusable workflow with separate worker, private-reader and TestOps-upload jobs. Each job uses only its own protected environment. Callers pass empty named secret placeholders and do not inherit secrets. - Reuse the original exporter and grade checks. Freeze each case's three Allure JSON files once. Allow known sibling results during per-case readback. Keep final exact selection coverage. - Submit each bundle at most once. Reconcile uncertain responses by readback. Retain only closed publication, failure and upload-intent receipts. A rerun attempt still fails the existing attempt-one admission guard. - Preserve failed original grades, failed cleanup and missing evidence. No missing original can become a passing result. - Keep external-input cases in the serial lane. Small native-only selections allow two cases at once. Small mixed selections allow one native and one external case. Larger selections retain eight native cases and one external case. - Add source, privacy and failure controls to the existing unit roster. Document the reporting flow and runner-group admission requirement. - Fix an existing corruption-test fixture that could write the same byte back to a random IV. Read and flip one bit instead. Keep the storage rejection and unpublished-plaintext assertions unchanged. ## Verification - 152 focused Node controls pass across the shared client, API, selection, CLI and streaming tests. The retained parent controls show the missing multipart and partial-readback behavior, plus the small external-selection error before its fix. - The focused workflow security control passes. Actionlint passes for both workflows. Changed JavaScript files pass syntax checks. Git diff checks pass. - Local controls use synthetic data and mocked HTTP. They make no provider call and do not start an app, database or native build. - Previous head `b51406df` passed PR CI (48 jobs pass and one is skipped). The review-driven test-only successor `0c65cc21` then failed one unchanged snapshot-storage corruption test, plus the final CI summary. The latest head includes its narrow fixture repair. Its first CI attempt separately hit runner shutdown/lost-communication failures, two setup/timeouts and a heartbeat-cancel cleanup HTTP 500/lock timeout; those observations are retained without claiming their underlying cause. One same-head failed-and-cancelled-only recovery now passes at `bf9c3e8f`: 48 jobs pass and one is skipped, including required `ci / verify` and `ci / e2e`. All 31 already-passing jobs retained their original execution records; all 15 recovered leaves and both dependent summaries passed. The first attempt remains recorded; the recovery does not establish the underlying causes of those failures. A small AWS run after delivery must prove that one case and both attachments are visible before another original case ends. That live proof is pending; local controls do not prove vendor ingestion or environment-secret resolution. - Retained earlier fixture-calibration failures remain recorded. The final integrated Node control run is green. A deterministic Node crypto/filesystem control also reproduces the no-op corruption mutation with a valid IV byte equal to `X`, then passes after flipping that byte. This control uses no app or database. The failing CI byte was not captured, so the exact historical byte remains unknown. ## Risks - The called workflow needs its exact master ref in the existing selected-workflow runner group. No IAM role change is needed. - Environment-only secrets must resolve in the called jobs. The caller passes no actual model or reporting key. The small live proof must confirm this behavior. - TestOps does not document an idempotency key for this upload route. An uncertain write is never resent automatically. If readback cannot verify it, the workflow fails with retained closed receipts. - The existing job run can already be marked finished by the selection watch. The live proof must record its actual stage and prove subsequent uploads are accepted. This PR does not reopen, stop or close that job run. - Backend ingestion does not prove that an already-open TestOps page refreshes automatically. - The corruption-test repair changes no production storage or cryptography. New-head AWS CI must qualify the existing database-backed owner test. - Each case adds short protected reader and upload jobs. The app source, original case clocks, graders, model, budgets, raw-evidence privacy boundary and zero eval retries stay unchanged. The versioned stream concurrency group serializes new streaming runs; an old immutable batch run can continue in its original group. ## Model Used OpenAI `gpt-6.1-sol`, with tool use and code execution. The session does not expose a context-window size. Independent source peers used the same model family. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 14:18:00 -06:00
DottaandPaperclip	946735d03a	Persist task sorting in URLs and show newest routine tasks first ... Save task sort choices in URLs across task views. Open routine tasks by creation time, newest first, with stable sorting before pagination. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>	2026-11-02 13:32:40 -06:00
DottaandPaperclip	1f8f2e195f	perf(ui): reuse parent metadata when loading the inbox (#18645) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The Mine inbox combines task rows with approval and failure items. > - It waits for parent captions before it reveals the initial list. > - It fetched rich diagnostics for every parent, including parents already in the task index. > - Parent captions only use fields that the index already provides. > - This change reuses loaded parents and fetches missing parent metadata through the index view. > - Failure items keep their rich interruption data and the initial reveal boundary stays in place. ## Linked Issues or Issue Description **What existing behavior does this improve?** The initial load of Mine and other inbox views that need parent task captions. **Subsystem affected** `ui/` collection reads. The API contract and database schema stay the same. **Current behavior** The inbox fetches all parent and failed-run task IDs with `view=rows` after the primary task request. This adds active-run, blocker, review, interruption and recovery work to the reveal path. The observed hosted case fetched four parents. One was already in the 31-row task index. The other three were completed parents. The serial context response took 1.0–2.9 seconds during the diagnostic reloads. **Proposed behavior** Reuse parents from the selected source index. Fetch missing parents with `view=index`. Keep `view=rows` for every failed-run task, including overlaps with parents or source rows. Keep both role-specific ID lists in the query key. Drop old placeholder rows when the new index supplies a parent. **Reason and benefit** This removes unnecessary diagnostic work from a required serial request. Five alternating same-four-parent requests on the hosted app had index times of 189, 162, 156, 128 and 169 ms, versus rich-row times of 282, 211, 329, 219 and 284 ms. All requests missed the response cache. All consumed parent fields matched. This isolates the projection cost; it is not a deployed UI speedup claim. **Breaking changes** None. Membership, ordering, nesting, archive state, authorization and failure presentation retain their owners and contracts. Refs: #18323 and #18492 for the prior collection and startup work. This change addresses the remaining parent-context read. ## What Changed - Partition missing parent metadata from failed-run context. Load the two batches concurrently, with at most two batch requests in flight. - Keep richer failure rows last in the context result. Filter placeholders by company and current requested IDs. Track completed failure reads, including missing tasks, so a parent placeholder cannot reveal an unverified failure card. Keep existing verified cards mounted through refresh. - Cover delayed captions, reuse of indexed parents, refreshed parent captions, failure overlaps, resolved failures and parent-to-failure cache transitions. - Add a real browser check for delayed parent metadata and stable geometry through deferred diagnostics. Give the test its own disposable company. - Add a reusable local timing harness and update the collection verification recipe. ## Verification - Baseline source: `0710c5684243a663c09ff907011ab88838173a02`. Final measured candidate: `c8c69b5af11966b4144707a3da21e840c73c6cca`. Production UI assets were built from the same checkout and dependencies, with only the Inbox module changed between variants. - Targeted regression failed on the old implementation and passed after the change. All 58 Inbox component tests pass, including existing archive/undo and order checks. - Full workspace typecheck passes. The final UI typecheck passes. Full build passes. Token gates, map checks and diff whitespace checks pass. - All 12 collection browser checks pass, including pagination and reading-position restoration. - The full CI workflow passes on `c8c69b5af11966b4144707a3da21e840c73c6cca`, including all server, chat, workspace, serialized, runner and E2E shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/41502803275). Greptile reports 5/5 on this commit with no unresolved threads. The separate Paperclip Review check is still queued. Its first attempt saved a text result but did not complete the GitHub assessment. Normal retry was rejected because that attempt already recorded a result. The current-head review request is queued. The review service is now unavailable, so this external gate remains open. - The duplicate local `pnpm test:run` was interrupted after the equivalent CI run passed. Its six chat shards passed 1,828 tests. The general-server phase was stopped. This local full-suite run is incomplete, not a passing result. - Local synthetic fixture: four parents and 30 children, with three completed parents archived. The fixture also has 30 archived child records. No agent or provider calls ran. Mac arm64, Node 24.22.2, Chromium, production assets, 1440×900, native CPU, loopback network, fresh HTTP/query caches for each cold sample. - Five alternating local pairs produced identical 31-row membership and order and no browser errors: | Journey | Baseline median (range), ms | Candidate median (range), ms | Median paired saving, ms | | --- | --- | --- | --- | | Cold first usable Mine rows | 310 (252–435) | 258 (250–261) | 50 | | Warm Recent → Mine | 60 (56–64) | 67 (61–71) | -7 | | Task → Back to Mine | 42 (40–47) | 43 (40–45) | 0 | The final cold result has a material control limitation. The median primary-request start was 259 ms for baseline and 210 ms for candidate. Much of the cold difference occurs before the changed request. The parent response medians were 19.3 and 13.5 ms, with fewer parent IDs requested. Do not attribute the full 52 ms difference between cold medians to this fix. Four cold pairs were faster and one was 6 ms slower. Warm Recent → Mine was about 7 ms slower at the median; Back was unchanged. These runs establish no warm-navigation gain. The earlier five-pair experiment showed the same startup confound and mixed warm/Back results. All samples remain in the local evidence. A hosted final-shape comparison timed out and was cancelled by closing its owned browser tab; it supplies no result. The candidate has not been deployed. Reproduce the contract and geometry check against an isolated production UI with `pnpm exec playwright test --config tests/perf/collections/playwright.config.ts parent-context.spec.ts`. Run `tests/perf/collections/measure-mine-context.ts` against baseline and candidate loopback production origins that share one disposable API, as documented in `tests/perf/collections/README.md`. Preserve upstream startup timings as controls. Required parent responses remain inside the reveal boundary. ## Risks - Explicit-ID `index` and `rows` requests use the same company, visibility and actor filters. They do not use the inbox archive filter. An archived parent is still returned by either projection. A deleted or unreadable parent is absent in both. A second index request for a failure-parent overlap would not restore missing caption data. - A failed-run task must never reuse parent-only metadata. The query key includes each projection role. Completed reads carry explicit failure-ID coverage. The tests cover overlaps, pending role changes, late heartbeat data, known absence, and transitions to empty views. - Old parent context must not override a fresh indexed parent. The current-ID placeholder filter and caption-refresh regression cover this case. - The hosted app still has unrelated server contention. This PR reduces parent request work. It does not claim to fix all hosted loading delay. Deployment acceptance should confirm the serving revision, compare matched Mine reloads and inspect errors and row stability. Revert this change if failure items or captions regress. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, browser diagnostics and code execution. The exact runtime model identifier and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 13:29:04 -06:00
DottaandPaperclip	5a07c30c1a	fix(workspaces): reduce temporary disk use during remote capture (#18644) ... ## Thinking Path > - Paperclip manages AI agents and preserves their completed work. > - Native remote finalization must save workspace files before it releases the writer. > - Portable capture copied the working tree and Git history, repacked the history, and then wrote an archive. > - These temporary copies can exhaust a small sandbox after normal development work. > - A completed provider turn then fails finalization and file preservation repeats the same failure. > - This change streams checked working files and keeps only an independent Git copy in scratch. > - The save still rejects changed sources and incomplete output. ## Linked Issues or Issue Description Related: #18403 added portable remote workspace storage. #18172 handles disk pressure in Daytona directory transfer. This failure occurs earlier, while the portable archive helper creates its source archive. That earlier repair does not cover this path. **What happened?** A completed native run failed workspace finalization three times. Its file-preservation retries also failed. In a retained 10 GB Linux sandbox with about 1.5 GB free, the deployed helper failed during `git repack -a -d` with `Out of diskspace`. The working files used about 603 MB, and Git used about 444 MiB. **Expected behavior** Capture a coherent workspace when free space can hold one full archive plus an independent Git copy. Preserve working files, staged changes, unpushed commits, and existing recovery gates. **Steps to reproduce** 1. Use a disposable Linux sandbox with a Git repository and generated working files. 2. Leave enough free space for an archive and a Git copy, but not a second working tree and another repack. 3. Run the portable capture helper. The old helper fails while creating temporary data. 4. Run the fixed helper against the same source and free-space limit. Verify each archived file hash. The small regression uses real file serialization and a controlled filesystem capacity failure at archive creation. It failed with `ENOSPC` before the production edit and passes afterward. **Paperclip version or commit** The observed staging server served `42c64d4ec4b9b0898556fcdf6ba304c4285a7819`. The repair is based on `a92f616cd0`. **Deployment mode** Hosted staging. Linux x64, Node 25.9.0, Daytona sandbox. Local tests use a macOS source checkout. ## What Changed - Stream working files from checked file handles. Keep only normalized Git metadata in scratch. - Remove the second repack after a mirror clone that already uses `--dissociate` and `--no-hardlinks`. - Check each streamed file's hash, size, mode, and filesystem identity. Recheck the source inventory and Git state before returning a receipt. - Remove incomplete archives after failure or cancellation. Keep pre-existing archive files intact. - Add space-limit, concurrent-change, cancellation, and existing-file safety cases. Update execution semantics and workspace verification recipes. ## Verification - Red before the production change: the capacity regression failed with `ENOSPC`. Green afterward: 54 archive and transport tests pass. - The existing tests restore ordinary, linked, and borrowed Git repositories after their sources disappear. They retain staged, unstaged, and unpushed work. Credential and path guards also pass. - Live comparison on the same retained sandbox: the deployed helper failed after 19.4 seconds with Git disk exhaustion. The fixed helper completed in 35.3 seconds. It produced a 1,081,872,803-byte archive with 24,111 entries. An independent reader checked every archived file hash and the manifest digest. The source commit stayed unchanged. Diagnostic scratch was removed. - This live probe exercised the changed capture boundary. It did not deploy a server, upload a production snapshot, finalize the failed run, or replay its provider turn. Full task recovery still needs rollout and the normal export recovery path. - `node scripts/check-maps.mjs` and `git diff --check` pass. - Current-head CI passes: [PR workflow](https://github.com/paperclipai/paperclip/actions/runs/41499985180), commit `d42700f7f4b0bef14a5969a44c2da0b2513e0ed2`. Greptile rates the same head 5/5 with no actionable findings or unresolved threads. - The independent Ponytail advisory also rates this exact head 5/5 with no findings. Its structured check remains queued because the automatic request was bound to the earlier head. A [fresh current-head review request](https://github.com/paperclipai/paperclip/pull/18644#issuecomment-6585296011) was accepted and queued. Its predecessor has a successful workspace export but remains active in finalization, so the new review cannot start yet. This review-service publication gate must complete before claiming every check is green. - `pnpm -r typecheck` and `pnpm build` pass. The final server-only typecheck also passes after the test additions. - Supported-runtime verification on Node 24.11.0: all 54 archive/transport tests and all 46 existing workspace lifecycle tests pass. The lifecycle suite uses a disposable real Postgres database and local filesystem transport with encrypted snapshot storage. It checks save/restore after losing the original allocation, ownership, retention, and file continuity. This is separate from the live Daytona capture probe. - The first extra lifecycle attempt could not start its local Postgres binary because the installed Darwin package lacked library aliases. Local dependency setup was repaired before the passing run. No production or repository code changed for that setup. - The full local `pnpm test:run` on the host default Node 26.4.0 stops in chat-test collection with `No test found ... in lines ...`; it has no suite verdict. A clean retry repeats the failure. The unchanged `a92f616cd0` test wrapper reproduces it with the same installed dependencies, after correcting an initial baseline setup that lacked built package entries. This is a local test-runner limitation, not a full local pass. Current-head CI remains the full-suite gate. - One focused run timed out while deeply comparing 4 MB buffers under concurrent build load. Complete SHA-256 comparisons retain the content check without that assertion cost. The capacity case then passes in under one second, and the final 54-test run passes. The disk allowance and production code did not change. ## Risks - Streaming reads the live source instead of a copied working tree. File identity, content hashes, mode checks, the final inventory read, and Git signatures must reject concurrent changes. Regression tests exercise these checks. - Capture still needs space for the full archive and normalized Git metadata together. It cannot export from a completely full disk. - Archive framing and restore validation stay compatible. The change has no database migration or new execution authority. - The incident remains unrecovered until the fixed server performs normal workspace finalization. No deployment or merge is included in this PR. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code editing, shell tools, browser diagnostics, and automated tests. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.4	2026-11-02 13:22:23 -06:00
DottaandPaperclip	026fd9e345	fix(server): keep controller identity independent of Git probes (#18649) ... Read the existing boot timestamp without synchronous Git diagnostics. Keep the voice-retention precision test clock fixed. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-02 13:11:41 -06:00
Chris KaubischandClaude Opus 5.5	b0f5451fdf	fix(claude-local): skip the 2.1.291 copy hint in the setup-token prompt parser (#15550) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - AI connections let a user connect a Claude subscription for all agents of an organization > - In a Docker deployment the server runs `claude setup-token` in a PTY and parses its output for the sign-in URL (`server/src/services/local-ai-browser-login.ts`, `packages/adapters/claude-local/src/server/setup-token-parse.ts`) > - The Docker image installs `@anthropic-ai/claude-code@latest`, and `claude` 2.1.291 adds a copy hint line between the URL and the code prompt > - The parser requires the prompt on the first non-blank line after the URL, so it returns null and the UI waits without an error > - This pull request lets the parser skip that exact hint line > - The benefit is that Claude subscription sign-in works again with the current CLI ## Linked Issues or Issue Description No issue exists. I searched open and closed issues and pull requests for "Preparing browser sign-in" and the setup-token parser. **What happened?** In the "Claude - Subscription" connection dialog, the UI shows "Preparing browser sign-in…" and "Start sign-in again". It shows no sign-in link and no error. "Start sign-in again" starts the same wait again. After 300 s the run fails with "Sign-in ended before the account was connected". **Expected behavior** The dialog shows the sign-in link and the code field a few seconds after the start. **Steps to reproduce** 1. Run the Docker image with `claude` 2.1.291 (`@latest` at the time of writing). 2. Open the AI connection dialog and select "Claude - Subscription". 3. The dialog stays on "Preparing browser sign-in…". **Paperclip version or commit** `master` at 03cf6a6ec (includes #14970). `claude --version`: 2.1.291. **Deployment mode** Authenticated, private exposure, Docker. **Relevant logs or output** The server logs nothing for this flow. A manual run in the container shows the output that the parser reads (URL values removed): ``` docker exec -it -u node <container> sh -c 'CLAUDE_CONFIG_DIR=$(mktemp -d) CLAUDE_CODE_OAUTH_TOKEN= ANTHROPIC_API_KEY= ANTHROPIC_AUTH_TOKEN= BROWSER=true claude setup-token' Browser didn't open? Use the url below to sign in (c to copy) https://claude.com/cai/oauth/authorize?code=true&client_id=…&response_type=code&redirect_uri=ht tps%3A%2F%2Fplatform.claude.com%2Foauth%2Fcode%2Fcallback&scope=user%3Ainference&code_challenge=… …&code_challenge_method=S256&state=… Hold Shift while selecting to use your terminal's native copy Paste code here if prompted > ``` `findBrowserCodePrompt` reads the first non-blank, non-URL line after the URL. Here that line is the copy hint, so the parser returns null. The runner never calls `onPrompt`, and `check()` returns `sign_in_required` with no `authorizationUrl` and no error. ## What Changed - `setup-token-parse.ts`: add `COPY_HINT_LINE_RE`, which matches only the exact copy hint (CHA-spaced words and a curly apostrophe are allowed). `findBrowserCodePrompt` skips a line that matches it. The prompt line rules do not change. - `setup-token-parse.test.ts`: add a test with the 2.1.291 layout (hard-wrapped URL, CHA-spaced hint, blank lines, prompt). ## Verification - `bun test packages/adapters/claude-local/src/server/setup-token-parse.test.ts packages/adapters/claude-local/src/server/setup-token-characterization.test.ts`: 71 pass, 1 skip, 0 fail. - The 2.1.291 output above returns `null` before the change and the URL plus prompt after the change. - The test "returns null when a code-like value sits on the line after the URL" still passes. Other prose between the URL and the prompt still fails closed. ## Risks - Low risk. The change skips one fixed line and binds no value from it. - A later CLI version can change the hint text again. The flow then hangs the same way until the 300 s timeout. An earlier error (for example, when no prompt appears after 60 s) and a `log` option for `runSetupTokenLogin` in `local-ai-browser-login.ts` would make this visible. This pull request does not include them. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (code search, shell, test runs). A human ran the container commands and reviewed the change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-02 13:00:20 -06:00
DottaandPaperclip	a92f616cd0	fix(db): attribute bounded captures to execution owners (#18634) ... ## Thinking Path > - Paperclip manages agent work through shared server and database services. > - High query volume can spend main-thread time on SQL construction and result decoding. > - The existing bounded response capture counts rows and bytes, but it cannot name the operation that executed a query. > - Bundled callers can be missing, and measuring every parsed JSON value adds work to the diagnostic. > - This pull request carries a semantic execution owner into each measured query and bounds response sizing. > - Operators can rank query volume by owner before choosing a performance fix. ## Linked Issues or Issue Description Refs #17504, #17522, #17537, #17564. **What existing behavior does this improve?** The opt-in `PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW` diagnostic. **Subsystem affected** Database diagnostics and existing server execution boundaries. **Current behavior** The capture groups by table, operation, and builder caller. It cannot distinguish execution owners that reuse a prepared query. Bundled deployments need source maps for caller locations. The capture visits every returned value and serializes parsed JSON again to estimate its size. **Proposed behavior** Version 2 captures use fixed semantic owners for HTTP requests, delivery queues, execution phases, and budget work. They count completed calls, returned rows, errors, and completion wall time. They sample successful results 1, 17, 33, and so on, with fixed cell and string/byte limits. Complex values remain explicitly unmeasured. Semantic ownership works without source maps. **Reason and benefit** A short census can identify which operations issue the most queries or return the most rows. It preserves each query's execution owner through pooled connections, concurrent operations, and prepared-query reuse. It reduces diagnostic work without claiming per-query CPU attribution. **Breaking changes** Opt-in capture output moves from `entry.v1`/`summary.v1` to `entry.v2`/`summary.v2`. Sampled primitive byte lower bounds replace all-result byte estimates. Consumers must use the new fields and must not extrapolate them into a total. There is no database migration or application API change. ## What Changed - Added `withDatabaseOperation(owner, action)` to the DB package. The server owns semantic labels. Queries retain execution ownership in their completion closure. - Added fixed owners at existing HTTP, delivery, budget, and execution timing boundaries. Delivery work replaces the context of the producer that woke it. - Added complete call/row/error counters and completion wall durations within the existing bounded window. Durations include waits and are not CPU measurements. - Sampled and bounded primitive-value sizing. Complex and oversized values remain unmeasured. Known owners skip repeated stack extraction. - Preserved the deadline, group limit, paced output, query outcomes, and existing coverage exclusions. Updated the operator documentation. - Added real PostgreSQL owner-isolation and prepared-query tests, server scope tests, and plain bundled-runtime proof. - Moved projection-test and benchmark full-response sizing into an explicit test helper. Existing payload-size thresholds remain unchanged; these synthetic checks do not use the production sampler. ## Verification - Final-commit DB capture suites: 43 tests passed in 13.86 seconds against the actual PostgreSQL driver, including concurrent owners, prepared reuse, transaction failure, a dedicated client, sampling, and a query crossing the deadline. DB typecheck passed. - Server attribution and existing delivery/execution/budget suites: 53 tests passed. Final concurrent HTTP GET/JSON POST scope refinement: 5 tests passed. - Runtime bundle suite: 8 existing tests passed. The new no-source-map owner test passed after its fixture was corrected to use a driver thenable. An ordinary native Promise can bypass an overridden `.then`; the real PostgreSQL tests separately cover driver integration. - Observer-only experiment: three alternating pairs per case on Node 24.22.2, macOS arm64, 3,000 calls per run after warm-up. For an already-materialized synthetic JSON result, median capture CPU fell from 82.37 to 1.87 microseconds/call. For a 100-row primitive result, it fell from 8.10 to 4.40 microseconds/call. Every result retained identity and all calls were counted. The new diagnostic intentionally measures less byte data; these are observer costs, not database latency or a hosted application speedup. Disabled-case timings were below one microsecond/call and too small for a performance conclusion. - Final lifecycle regression: 28 unit tests and DB typecheck passed. Closing the last capture disables and replaces owner storage; overlapping captures retain it. - Final-commit full recursive typecheck passed in 38.38 seconds and full build passed in 40.81 seconds. The build stamp matches `83ed4ba9cebd68d48929c46f15ab32d5c0b9687b`. Focused server attribution tests also passed 5/5 before the test-helper-only migration. Full test acceptance will use the repository's parallel CI gates; the unsharded local command takes hours. - Verification failures retained: initial sandbox runs could not open local IPC/PostgreSQL listeners; permitted reruns exercised the real services. One old test expected JSON serialization and was updated to assert explicit unmeasured complex values. The first bundle fixture used the wrong promise boundary, as described above. A retained-callback regression also demonstrated that disabling and reusing the same AsyncLocalStorage instance could resurrect an old owner; replacing the instance after the last capture closes fixes that case. - The first CI run found older projection tests consuming the previous full-response fields. Those consumers now use the full-response test helper, preserving their byte thresholds. The two complete affected route suites plus helper tests passed 115/115 in 73.08 seconds. A temporary full-row query negative control then failed the unchanged 2,048-byte limit at 5,000,812 bytes and was restored; five other service suites passed 34/34 and the affected chat cases passed 3/3. All three previously failing CI shards passed on the corrected head. A local skill-list benchmark smoke check completed 255 captured queries with no in-flight queries or overflow; this was integration verification under concurrent test load, not a timing benchmark. - Two final-head runner jobs lost communication with GitHub before reporting a test result. One targeted retry of those jobs and their dependent aggregate passed. All 53 final-head checks passed, with 3 intentional skips; previously passing suites were preserved. - Hosted capture remains pending until the reviewed revision is deployed and a bounded window is explicitly configured. Reproduce the focused checks with Node 24 and the repository pnpm version: ```sh pnpm exec vitest run packages/db/src/response-capture.test.ts packages/db/src/response-capture-postgres.test.ts pnpm --dir server exec vitest run --config vitest.config.ts src/__tests__/database-operation-attribution.test.ts scripts/bundle-runtime.test.mjs pnpm -r typecheck pnpm build ``` ## Risks - Owner context is active while a valid future capture window is armed, up to 30 minutes before accounting starts. No-window and expired calls take the direct path. Diagnostics still add bounded work while enabled. - A nearest nested owner is reported. Broad `http.request` and unattributed groups remain visible; this does not identify every service callsite or SQL shape. - The existing census excludes tagged/raw explicit-execute/streaming paths and internal protocol commands. Its counts are not PostgreSQL transaction totals. - Completion wall time includes queue/database/network/scheduling waits. Concurrent durations overlap. Sampled bytes exclude complex values, large strings, and values beyond the sizing limits. - Activation is startup-only. Operators must choose the window after artifact preparation, allow rollout/warm-up time, and remove the expired setting afterward. ## Model Used OpenAI Codex, GPT-6 family, with code execution, repository inspection, and parallel agents. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 12:12:21 -06:00
DottaandPaperclip	6aabcff7d1	fix(ui): add space above failed attempt notices (#18636) ... Add token-based top padding outside failed-attempt notices and assert the full 16px gap in the browser check. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-02 12:10:43 -06:00
DottaandPaperclip	5f0bb518b0	fix(runner): handle blank assigned-tool schema references (#18637) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents discover assigned app tools through a governed gateway. > - OpenCode receives search and call tools instead of the full app catalog. > - A blank optional schema reference selected schema lookup instead of search. > - Agents received an unknown-tool error and could abandon valid app access. > - This pull request treats blank references as search and explains invalid schema requests. > - Agents can find their assigned tools while the gateway keeps its access checks. ## Linked Issues or Issue Description **What happened?** An agent called `paperclip_search_assigned_tools` with `{"query":"github","schemaTool":""}`. The server tried to find a tool with an empty name. It returned `assigned_mcp_tool_unknown` instead of search results. The agent then reported that its GitHub tools were missing. **Expected behavior** A blank optional schema reference must permit normal tool discovery. A bad schema reference must explain how to search again. **Steps to reproduce** 1. Assign an app tool to a native agent that uses on-demand discovery. 2. Search with `{"query":"github","schemaTool":""}`. 3. Observe the unknown-tool error instead of the assigned tool. **Paperclip version or commit** Reproduced from source before this fix, on master at d26628e35a. **Deployment mode** Native OpenCode with assigned MCP tools. The shared search code also serves other native providers. Related implementation: Refs #17740. A search for `schemaTool` found no duplicate issue or pull request. ## What Changed - Treat empty and whitespace-only schema references as normal search. Accept a zero schema offset in this mode. - Explain how to correct unknown references and positive offsets without a reference. - Test discovery, invocation, and revocation through the authenticated OpenCode MCP bridge. - Update tool descriptions, developer guidance, and the execution maps. ## Verification - The three new regression cases failed before the fix. - `pnpm exec vitest run server/src/services/native-runtime/assigned-mcp-tools.test.ts`: 43 tests passed after the fix. - Existing tests cover schema chunks, work-mode restrictions, revocation, approvals, and pinned tool catalogs. - `node scripts/check-maps.mjs`: passed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Full CI passed on commit `26d20fc19d846d46d7656a9f0160db5e81479dbc`: 55 successful checks, one neutral review check, and three skipped checks. Nothing is pending or failing. This includes all CI test shards, Runner verification, build, typecheck, security checks, and the canary dry run. - The duplicate local `pnpm test:run` completed its first five chat integration shards without failures. I stopped the remaining local run after the complete CI suite passed. The full local suite did not finish; the complete-suite result is from CI. - Greptile and the Paperclip code review each gave commit `26d20fc19d846d46d7656a9f0160db5e81479dbc` a 5/5 score with no findings. There are no unresolved review threads. - The initial focused test could not collect tests until the fresh worktree built its plugin SDK dependencies. `pnpm --filter @paperclipai/plugin-sdk ensure-build-deps` resolved that setup requirement. ## Risks Low risk. Blank references and zero schema offsets now select normal search. Nonblank references still require an exact authorized tool name. Discovery and calls keep their existing gateway checks. The change needs no database migration. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, tool use, and code execution. The host does not expose the exact backend model ID or context window size in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 12:07:28 -06:00
DottaandPaperclip	60a99307b3	fix(server): settle cleanup after reusable sandbox destruction (#18633) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A web-server replacement must preserve work and let completed cleanup become idle. > - Several task turns can use one remote sandbox through recorded lease handoffs. > - The last owner can destroy that sandbox while an earlier owner still waits for its own termination receipt. > - A completed agent-file cleanup can also retain a retry deadline from a competing callback. > - This pull request resolves confirmed destruction through the existing handoff contract and settles obsolete cleanup retries. > - The benefit is correct cleanup recovery without deleting history or weakening sleep checks. ## Linked Issues or Issue Description Refs: #18554. Related work: #14695 handles warm agent-file ownership; #13291 handles interrupted conversation continuation. This PR fixes cleanup recovery after confirmed final destruction. It does not change provider process reuse or start a new run. **What happened?** After ten qualified staging web-server replacements, a bounded database-idle check failed. Completed native runs still owned pending agent-file cleanup. Normal workspace archive confirmed final sandbox destruction. An earlier lease in the same recorded handoff chain remained unresolved. Two completed cleanup receipts also kept stale retry deadlines. **Expected behavior** Resolve an earlier owner's cleanup only when the complete recorded handoff chain leads to an exact provider-confirmed destruction receipt. Clear a retry when its cleanup is already complete. Keep real or uncertain work pending. **Steps to reproduce** 1. Complete a remote run, reuse its sandbox for a second run, then archive the completed workspace. 2. Recover agent-file collection for the first run after the final lease confirms destruction. 3. Separately, reconstruct a completed cleanup receipt that still has a retry deadline. 4. Observe that both records remain unfinished without this fix. **Paperclip version or commit** Observed on `1a6017d6460267ad89d7e1784e2b521c28d8e9ac`. The changed source was unchanged in master `97c4dc969e13364464765ffbde74955005bf44d3`, which is this branch's base. **Deployment mode** A managed ephemeral web server with PostgreSQL, object storage and a remote sandbox. Regression tests use local PostgreSQL and no live provider or model calls. ## What Changed - Resolve remote termination at read time through the existing strict reusable-lease handoff validator. Require one final owner and a valid destruction receipt. Preserve historical lease rows. - Use the same proof in agent-directory cleanup and remote-stop inspection. Keep stopped, ambiguous, broken and cross-company chains blocked. - Reconcile completed cleanup receipts with stale retry deadlines through the existing deadline worker. Compare the receipt before clearing a deadline. - Add recovery, refusal, restart quietness and concurrent-obligation regressions. - Document the cleanup contract and preserve deployment evidence, failed checks and remaining acceptance work in the existing plan. ## Verification - Both owning service suites passed 169 tests before the final receipt comparison was added. - The seven-case selection passed again after the release-race review fix: `pnpm exec vitest run server/src/__tests__/agent-directory-working-copies.test.ts -t 'stale retry|destroyed successor chain|stale-deadline settlement'`. - Repository-wide `pnpm -r typecheck` and `pnpm build` passed on final review fix `7935d99bed`. The full `pnpm test:run` is still running. Latest-head CI has 55 successful checks, three skips, and one neutral review result. Greptile is 5/5. The Paperclip reviewer’s race finding is fixed; its optional query-deduplication suggestion is documented and the fresh lock-boundary checks are retained. - The reviewer identified an early return after a lost receipt comparison. The strengthened race regression failed before the fix and passed afterward: release now acts on the returned authoritative row, completes the new cleanup obligation, and preserves canonical files. - The first focused attempt failed the new stale-deadline test because the public release guard skipped settlement. That guard was fixed. The original failure is retained. A sandboxed typecheck could not fetch pnpm signature metadata; the normal network-enabled command passed without changing verification policy. - Prior deployed evidence: ten Railway replacements, 12,521 reads, 1,566 verified saves, 34 matching file hashes, no HTTP failures, actual browser reconnects of 1.16–1.29 seconds, and one remote execution retained across replacement. Two harness false alarms were separately adjudicated without redeploying those steps. - That deployment evidence predates this PR. Current-image natural idle, automatic sleep/wake browser return and the isolated production canary remain unverified. Production has not changed. ## Risks - Cleanup authority must not cross a company or allocation boundary. The existing validator checks the entire recorded chain; stopped successors and unexpected owners remain unresolved. - Confirmed destruction permits no-save cleanup. It does not claim that uncollected agent files were saved. - Read-time resolution adds a scoped lease-history query only for expired reusable leases without direct termination proof. No new recurring worker or polling cadence is added. - The deadline update must not erase new work. It requires the same cleanup receipt and preserves concurrent changes. - No schema or public API changes are included. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, code execution and test tools. The precise backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:53:44 -06:00
DottaandPaperclip	0710c56842	fix: retry saved chat input after a successful run (#18621) ... ## Thinking Path > - Paperclip manages AI agents and the work they receive. > - Connected conversations save incoming requests before they can run. > - A request can remain queued when the previous turn finishes successfully. > - Task Retry sent that case through the failed-run retry guard, which rejected the successful run. > - This change resumes the saved request through its existing admission owner. > - The operator can continue saved work while preserving the prior result and current permissions. ## Linked Issues or Issue Description **What happened?** Task Retry returned a conflict for a restricted connected conversation when its latest run succeeded and newer input remained saved. A GitHub review can reach this state when a new PR head arrives during the previous review. The prior review cannot publish against the new head, but its completed turn is not a failed run that can be replayed. **Expected behavior** Retry should resume the saved request with its original source identity and current authorization. It must preserve the previous successful result and must not replay that turn. **Steps to reproduce** 1. Start a review through a restricted GitHub endpoint. 2. Send a PR synchronize event while that review runs and retain its deferred input receipt. 3. Let the original run finish successfully with the task still open. 4. Click Retry. Before this change, the failed-run guard rejects the request. With this change, the saved input creates one successor through normal admission. Related work: #17725, #16965, #15033. ## What Changed - Select the newest saved input by its canonical chat action and wake receipt. - Store its receipt ID in the task retry intent and resume it through the existing chat owner. - Preserve caller permissions, source authorization, task state, agent state, cleanup waits, and duplicate-click handling. - Exclude saved-input intents from generic board-message replay. Keep board messages in separate queued receipts and allow independently saved input after successful board turns. - Add signed webhook, HTTP endpoint, accepted-result preservation, and refusal tests. Update execution semantics and capability maps. Stabilize the existing Teams lifecycle test with explicit database result ordering. ## Verification - Current-head [CI passes](https://github.com/paperclipai/paperclip/actions/runs/41482727909): 54 successful checks, one neutral review, three expected skips, and a successful Snyk status. Greptile gives commit `53f02e9105` 5/5 with no unresolved review threads. - Reproduced the original HTTP 409 before the fix. Reproduced both review findings with three failing cases before fixing them. - Passed 25 focused and related GitHub retry cases, 52 ordinary queue tests, and three native startup cases against disposable PostgreSQL. Earlier related verification also passed 83 GitHub retry cases and 109 recovery/receipt cases. - Passed `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and the module boundary check. The final server typecheck passed after the review fixes. - The full local `pnpm test:run` attempt stopped on an existing Slack queue deadline assertion with a 1 ms database/host clock difference. That case passed in isolation. A second attempt was stopped when review changes superseded its source. Full local completion is not claimed; all chat, server, browser, build, and runner CI jobs passed on the final commit. - The runner CI job needed one rerun after a two-second startup timing assertion failed. That case and its two related cases passed locally; the rerun passed without code changes. - Live GitHub review publication with this patch is not verified. The signed provider fixture uses mocked GitHub responses. This branch is not deployed. ## Risks - This changes how Retry dispatches deferred external input after a successful run. It relies on the existing inbound owner to recheck source authority before execution. - Tests cover paused connections and agents, changed source bindings, completed tasks, pending cleanup, duplicate clicks, and preservation of the accepted result. - No schema migration or public API shape change is required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:46:46 -06:00
DottaandPaperclip	d26628e35a	test(server): stop voice report waits from crossing refresh deadlines (#18628) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Voice calls retain reports that background work refreshes for a finite period. > - Tests must check backlog processing without starting later refreshes. > - The backlog test advances fake time while real database work is still running. > - On a slow runner, this starts refreshes and breaks the expected request count. > - This pull request advances time only to scheduled passes before the first refresh. > - The test can check each report once and still prove that the worker becomes quiet. ## Linked Issues or Issue Description Refs #18607. This is a separate test failure from that PR's task-list avatar changes. The test was added in #17586 and is unchanged by #18607. **What happened?** [Server shard 5/12](https://github.com/paperclipai/paperclip/actions/runs/41472540522/job/125369899753) failed in `voice-call-reports.test.ts`, in "bounds report reconstruction and resets a partial scan when the same owner restarts". The test expected zero active work but found two active holds. Its logs contain 170 requests for 105 sessions, including 65 second attempts. The final `ci / verify` failure follows from this failed shard. **Expected behavior** Process all 105 reports once. Wait for each pass to settle before advancing to another scheduled pass. Keep the clock before the first 60-second refresh deadline. **Steps to reproduce** 1. Run the named test against a disposable PostgreSQL database. 2. Temporarily delay the first provider response by 3,250 ms with `node:timers/promises`. 3. Use the same 15-second wait budget for the old and new versions. 4. The old polling makes 315 requests across three attempts per session and fails. The new polling makes 105 requests, one per session, and passes. **Paperclip version or commit** Observed at PR head `7f1d137a53489e2a45ba05e506b5ba17b51d5882`. **Deployment mode** GitHub Actions. Local verification uses disposable PostgreSQL 17. ## What Changed - Keep fake time fixed while a report pass waits on database work. - Advance to the coordinator's next scheduled pass only before the first report refresh. - Disable implicit fake-time advancement in the assertion polling. - Allow 15 seconds for the 105-report wait and 30 seconds for the full fixture test. - Retain all assertions for bounded reconstruction, restart, exact session IDs, one attempt per report, and no later database or provider activity. ## Verification - Controlled comparison with the same injected delay and 15-second wait: old polling fails with 315 requests; fixed polling passes with 105 requests. The temporary delay is removed from this change. - All 52 tests pass across `voice-call-reports.test.ts`, `scoped-deadline-work.test.ts`, and `delivery-work-coordinator.test.ts`. - `git diff --check` passes. - The initial embedded PostgreSQL run could not start because its macOS package lacks a `libzstd.1.dylib` alias. The passing tests use a separate disposable PostgreSQL 17 cluster. - The full 15-test voice-report suite passes again after rebasing onto current master. - `pnpm -r typecheck` passes. - `pnpm test:run` was started, then stopped during chat-suite startup after no tests completed in that invocation. It has no complete result. Its cause is not established. - `pnpm build` passes. - Greptile gives 5/5 with no inline findings on `caa6db568deb9df98437d1a532cc345e9f5b16dc`. Its check completed successfully for that exact commit. - The first CI attempt passed all 12 server shards, including the voice-report regression. Three other jobs were interrupted: Runner Vitest 2/2 received a runner shutdown signal; Typecheck + Release Registry and E2E shard 5/8 have GitHub annotations reporting lost runner communication. These are incomplete checks, not passing retries. [First attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/1). - The single retry passed for all three interrupted jobs. The final `ci / e2e` and `ci / verify` gates passed. [Successful second attempt](https://github.com/paperclipai/paperclip/actions/runs/41488319181/attempts/2). - Final commit `caa6db568deb9df98437d1a532cc345e9f5b16dc`: all 56 reported checks are terminal and clean. The two optional Storybook checks are skipped. GitHub reports the PR as clean and mergeable. ## Risks - Low risk. The change affects one integration test. Production code and report refresh behavior are unchanged. - The test still has a finite wall-clock timeout. A database that cannot process the fixture within that budget will fail. ## Model Used OpenAI GPT-6 through Codex, with repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:39:37 -06:00
DottaandPaperclip	40b8075916	feat(ui): show scheduled task waits as Monitoring (#18624) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task indicators tell an operator whether work is active, waiting, or complete. > - An agent can yield until a saved monitor checks CI or another external result. > - The task currently keeps the In Progress icon between those runs. > - Changing the stored status would affect monitor admission and recovery. > - This pull request adds a read-only Monitoring display state with a blue clock. > - Operators can see and filter scheduled waits while execution keeps its existing rules. ## Linked Issues or Issue Description **What existing behavior does this improve?** Task status indicators, task collection filters, board columns, and search facets. **Current behavior** A task that waits for a scheduled monitor shows the same In Progress icon as active work. **Proposed behavior** Show Monitoring with a static blue clock when an agent-owned task has a saved ordinary monitor and no active run, execution lock, review participant, or active recovery action. Return to the normal task indicator when the check starts. Preserve review, blocked, terminal, and quota-recovery states. **Reason and benefit** Operators can distinguish scheduled waits from current agent work without reading the task history. **Breaking changes** None to stored task statuses or execution rules. Task reads gain optional displayStatus. Collections and search gain a separate read filter. Existing status filters keep their lifecycle meaning. Related historical work: #4396 covered a monitoring label exemption. This change uses the saved monitor and a read projection. ## What Changed - Add one SQL projection for task detail, compact lists, summaries, relations, search results, and display-status counts. - Show the blue clock in shared task icons and chips, task detail and breadcrumbs, properties, references, list rows, recent navigation, and agent task cards. - Add Monitoring to task filtering, grouping, sorting, board columns, and search facets. Prevent a drag from creating this derived state. - Keep stored lifecycle statuses, monitor scheduling, checkout, recovery, permissions, and agent status filters unchanged. - Add database and UI regression coverage, full-page Storybook examples, and product and access-map documentation. ## Verification - Passed `pnpm -r typecheck`. - Passed `pnpm build`. - Passed `pnpm check:token-gates` and the map/catalog structural checks. - Ran the new database suite on a disposable PostgreSQL 17 database. It covers read surfaces, filter pagination/counts, queued/running checks, rescheduling, completion, retained monitors, review, and recovery holds. - Ran existing search, list, board, sidebar, task status, and filter tests. Corrected board queries use displayStatus and eight columns. - The full local `pnpm test:run` stopped in a chat integration shard: 12 failures (four timeouts and eight FIFO publication assertions). That run spanned a rebase. A focused rerun on the current source passed all 21 selected cases without assertion or timeout changes. The original cause is not established; this does not claim the full local suite passed. Final-head required CI passed (ci / verify and ci / e2e, run 41488267209). - Passed 82 tests for task API summaries, recovery precedence, search, and optimistic updates; 54 tests for status cards, database reads, and board moves; and 27 tests for blocker relations and recent-task cache updates. One intermediate database setup hit its 20-second hook timeout; the serial rerun passed. - Greptile is 5/5 on d003ea672d. All review threads are resolved. The separate optional Paperclip Review ended early and returned neutral; GitHub rejected a direct rerun request with HTTP 404. It is not counted as a completed review. - Review fixes cover escalated recovery, is:open, optimistic status changes, blocker-summary projection, and recent-task dedupe. - Reviewed 15 screenshots from production pages and components with isolated Storybook API fixtures: task detail/properties/breadcrumb, list, groups, status filter, board, search/results/filter counts, reference pills, sibling navigation, agent cards, picker, dashboard, inbox, profile, watched status-card tasks, light theme, and mobile. This does not claim deployed staging verification. ## Risks - Read projections must stay consistent across full, compact, summary, and search responses. Tests cover those paths. - Monitoring is not a writable lifecycle status. The board does not accept drops into its Monitoring lane or onto a monitored card. This prevents a drag from implying a scheduled check. Move work into In Progress through that lane instead. - The read projection adds run/recovery existence checks. Existing heartbeat_runs_company_issue_created_idx and issue_recovery_actions_company_source_status_idx indexes cover the company/task lookup. It adds no worker, timer, recurring query, migration, or provider call. No production load benchmark is claimed. - Older clients can ignore displayStatus. Existing lifecycle status filters keep their behavior. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository tools, code execution, and browser tools. The exact backend model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:35:30 -06:00
DottaandPaperclip	b9e94b85a6	feat(observability): trace Daytona lease acquisition and resume (#18626) ... ## Thinking Path > - Paperclip manages work performed by AI agents. > - The server prepares a remote execution environment before the agent can respond. > - OpenTelemetry traces show the time spent in each preparation phase. > - A retained Daytona sandbox can take minutes to resume from an archive. > - The provider worker previously exposed no child timings for this operation. > - This pull request adds bounded lease phase spans under the existing worker trace. > - Operators can separate provider startup from lookup, teardown waits, and workspace verification. ## Linked Issues or Issue Description **What existing behavior does this improve?** Daytona environment acquisition and resume tracing. A long worker invocation currently hides the operation that consumed its time. Related tracing foundation: #10999. No duplicate open PR was found. **Subsystem affected** The Daytona sandbox provider and the server's existing plugin span validation boundary. **Proposed behavior** Show lookup, start or recovery, teardown waits, workspace verification, and fresh acquisition setup as timed child spans. Group verification steps beneath one parent. Record the bounded provider state before start, including archived. **Reason and benefit** An operator can identify which step delayed the first agent response. SDK startup includes its internal readiness wait; this change does not claim to measure provider queue time separately. **Breaking changes** The opt-in trace schema gains fixed span names and one closed state attribute. Execution, authorization, resource ownership, deadlines, and retry policies keep their existing contracts. ## What Changed - Add lease phase spans through the existing invocation tracer. - Add the fixed span names and state values to host validation. Define the state key in the shared startup attribute contract. - Preserve the exact work result or error if tracing fails. Never repeat work through the tracing wrapper. - Test worker receipts for archived and warm resume, recovery, concurrency, failure, disabled tracing, and cleanup. - Document the waterfall and its attribution limits. ## Verification - Daytona plugin suite: 525 passed, 7 skipped. The expanded tracing suite separately passes all 12 cases. - Host span validation and instrumentation tests: 46 passed, 3 skipped. - Standalone Daytona TypeScript check passes. - Module boundary and whitespace checks pass. - Full recursive typecheck and full build pass. - The full local Vitest attempt at b007a8e64f stopped in chat shard 1: 337 passed, 2 failed, 1478 skipped. One retry-notice case exceeded 15 seconds. The next case then reported that fake timers were no longer installed. A focused rerun of all four retry-notice cases passed. This is not a full local suite pass. - Final head 8e1f81523b passes CI and Greptile 5/5, with no unresolved review threads. The CI chat shards pass despite the preserved local timeout attempt. - The expanded lease tracing suite passes 15 tests; host/shared tracing suites pass 66 tests. Disabling the wait flag makes the new teardown test fail, and restoring it passes. Standalone plugin typecheck passes. - The tests drive the real plugin worker RPC and invocation tracer with a controlled provider fixture. They validate receipts and host parent IDs. They do not establish deployment or Honeycomb ingestion. Inspect a new deployed run to verify the exported waterfall. ## Risks - A new provider phase must be added to the closed host allowlist or it appears as other. - SDK-internal start and readiness work remains one measured operation. - Older workers without withSpan run normally without these new detail spans. - Receipt budgets and clock validation can omit detail from very large or invalid receipts. ## Model Used OpenAI Codex, GPT-6, with repository tools and code execution. The exact runtime model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.3	2026-11-02 11:32:03 -06:00
DottaandPaperclip	97c4dc969e	test(evals): retire duplicate chat TestOps plans (#18625) ... ## Thinking Path > - Paperclip manages AI agents and preserves their work history. > - Product E2E cases use TestOps plans to select real AWS evaluations. > - The full chat catalog now includes the former smoke and completion-pilot cases. > - The two old bootstrap workflows can recreate their smaller plans after deletion. > - This pull request removes those workflows and stops their plan-creation functions before API access. > - The full-catalog workflow keeps its shared clients, evidence rules, case identities, and measured results. ## Linked Issues or Issue Description Refs #18542, #18516. The user requested removal of the duplicate smoke and completion-pilot plans and investigation of their recreation paths. Both old workflows accepted manual dispatch only. Their bootstrap operations created plans by name when the plans were absent. They had no schedules in their workflow definitions. The old workflow records were disabled before plan removal. An existing pilot execution was allowed to finish successfully. This change does not cancel executions or delete cases, results, launches, or private evidence. ## What Changed - Remove `chat-feature-testops.yml` and `chat-completion-testops.yml`. - Make the former singleton plan creator reject every call before API reads or mutation markers. - Reject pilot bootstrap before API access. Keep the full-catalog branch and its integration validation. - Remove tests that require the retired workflow files. Keep shared helper and full-catalog controls. Run the trusted-checkout boundary test against the active catalog workflow. - Replace old setup instructions with retirement guidance and the active full-catalog route. The master sync preserves the incoming typed launch-search fix from #18620, including both regression tests. The only conflict was the retired pilot documentation. Its search-format note remains as a retained-helper contract. ## Verification - Two new retirement controls fail against the parent code because it reaches an API or mutation marker. Both pass after the change. - Six owning Node suites after the master sync: 146 passed (144 before the sync). Shared contract: 22 passed. Catalog and retained export controls: 12 passed. Focused workflow and release controls: 7 passed. - Two modified API modules pass Node syntax checks. `git diff --check` passes. - The pure workflow-security file has the same four failures and 13 passes on both the untouched parent and this branch. Two assertions expect eight trusted bootstrap steps where master now has nine. Two lists omit the already-shipped catalog workflow. This change does not repair or relax those unrelated assertions. - A broader release-test invocation had 73 passes and 20 sandbox failures from blocked mock HTTP listeners or npm permissions. Its focused affected workflow control passes. A standard Vitest setup attempt stopped on a missing `acpx/runtime` bundle dependency before tests. No local app, database, provider, or native runtime was qualified. - Current-head PR CI passes: 48 successful jobs and one skipped job, including both required checks. Greptile is 5/5. Independent source review passes. No new model run is needed for this retirement. ## Risks - Existing integrations that point at either retired workflow will stop dispatching. This is the requested behavior; use the full catalog instead. - Historical evidence readers still use shared helpers. Those helpers remain, including the transport and official-context parser used by the active catalog. - Workflow removal is a repository change. TestOps plan deletion and disabling obsolete executors are separate service actions. Neither action should delete shared cases or measurements. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with source editing, shell tools, and independent source review. The context-window size is not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:07:32 -06:00
DottaandPaperclip	3f7811eef0	fix(runner): keep requests responsive during retired backup cleanup (#18619) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Remote native runs keep durable harness backups for continuation and recovery. > - Publishing a new backup retires earlier backup directories in the server process. > - The synchronous permission walk and recursive removal block unrelated requests. > - This pull request makes that cleanup asynchronous and keeps publication waiting for completion. > - The benefit is continued request progress while large retired backups are removed. ## Linked Issues or Issue Description **What happened?** Publishing a backup with many small files blocks the Node event loop during directory permission repair and removal. A local comparison with 20,000 retired files reproduced 1.37–1.95 second event-loop stalls. This is a pause candidate. A short hosted profile has not established it as the dominant source of sustained CPU use. **Expected behavior** Unrelated requests can progress during cleanup. Publication still waits for complete removal, preserves rollback on a failed lease stamp, and leaves live files and symlink targets unchanged. **Steps to reproduce** Run the benchmark in `doc/performance/native-backup-publication.md`. It compiles the actual publication service from a pinned baseline and the working tree, prepares immutable retired backups, and sends foreground HTTP probes from a separate process. **Paperclip version or commit** Measured baseline: `774ddc72713d6aa7cb516691c71f1fc23bcfa5ff`. The measured treatment source is committed in this branch. Current head: `0a69cf2d43c2f9fcf032d96c1dceb56163bf21ba`. The runtime patch is unchanged from the measured source; later commits only strengthen the test and benchmark evidence paths. **Deployment mode** Local compiled service on macOS arm64 with Node v24.22.2. No hosted rollout has occurred. Related work: #18380 changed retention scheduling. #13676 included the existing immutable-backup cleanup behavior. Neither change makes publication cleanup asynchronous. The GitHub duplicate search also found #11929, which concerns database dumps rather than native harness backups. No matching open event-loop cleanup issue was found. ## What Changed - Use asynchronous filesystem operations for retired-backup cleanup. Visit directories serially and skip unnecessary metadata calls for file entries. - Await cleanup before publication proceeds, during rollback, after successful rotation, and in executor finalization. - Exercise immutable current and previous backups in the publication test. Preserve the existing symlink-isolation and rollback tests. Add a real separate-process HTTP probe that must complete while publication is still active. - Add a repeatable service benchmark and document its fixture, metrics, and acceptance limits. ## Verification Draft status: focused correctness tests, the local comparison, and repository typecheck pass. The complete local test gate is incomplete. The build passed. Complete CI, hosted acceptance, and final-head review remain pending. - Final focused Vitest run: 2 files, 4 tests passed in 649 ms with the normal server configuration. The regression warms a separate Node client, waits for its HTTP request during the stamp callback, queues the response for the next event-loop turn, and then allows final retired-tree removal to start. The async treatment serves that response before publication completes. The original synchronous cleanup fails the same final regression in a separate source fixture. That expected control failure is retained. - Syntax and `git diff --check` passed. - `pnpm -r typecheck` passed on head `332c57b88c` in 152.18 seconds, including the native Runner build. Validation uses Node v24.22.2 and the repository-pinned pnpm v9.15.4 with isolated dependencies. - `pnpm test:run` was intentionally stopped after 1,390.24 seconds (exit 143). All six chat shards passed, covering all 1,817 parameterized cases across their exact partitions. The general server phase was advancing but incomplete. The repository timing manifest estimates another roughly two hours for the serial remainder, so validation moved to final-head parallel CI. This is not a full local test pass. The command began on `332c57b88c`; later changes only refine the regression and benchmark, and the final regression has its own focused pass. - Benchmark failure/metadata checks pass in a tiny isolated Git fixture: an injected worker exit 17 is reported in 0.44 seconds without hanging, and the result includes a staged source change when the unstaged diff is empty. - `pnpm build` passed in 41.80 seconds on `9f7e4e669b`. The final commit only releases a pending test probe on failure; runtime and build sources are unchanged. An injected client exit verifies that final test failure path exits promptly in 0.97 seconds instead of hanging. - Three alternating baseline/treatment pairs used fresh fixtures with 100 directories × 100 files in each of two retired backups. Each publication removed 20,000 files. A separate process sent HTTP probes at fixed 5 ms intervals, including overdue arrivals. - Every run verified the replacement manifest, complete cleanup, and unchanged symlink-target contents and mode. All 2,395 HTTP requests completed without errors. The largest measured generator delay was 8.8 ms. | Metric | Baseline median (range), ms | Treatment median (range), ms | | --- | ---: | ---: | | Foreground request p95 | 1,357.7 (1,287.4–2,014.0) | 1.8 (1.7–25.9) | | Maximum event-loop delay | 1,439.7 (1,371.5–1,954.5) | 19.9 (17.4–40.7) | | Publication completion | 1,429.1 (1,360.2–1,952.9) | 1,439.5 (897.9–1,649.6) | | Process CPU during publication | 1,194.1 (1,041.8–1,225.5) | 2,423.1 (1,995.7–2,514.7) | The treatment served 180–329 requests while publication was active. The baseline served zero during that interval. Process CPU includes foreground work that the baseline cannot serve until cleanup completes. The result supports responsiveness, with a CPU tradeoff. It does not establish lower total CPU or a hosted latency gain. Reproduce with Node v24.22.2: ```sh pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-harness-backup-cleanup.test.ts src/services/native-runtime/native-harness-backup-publication.test.ts node server/scripts/benchmark-native-backup-publication.mjs --baseline-ref 774ddc7271 --output /tmp/native-backup-publication.json ``` The benchmark records the baseline SHA, treatment HEAD and source diff. Raw local evidence remains outside the repository. A first small smoke run hit a sandbox restriction on loopback listeners; the authorized retry resolved it. The smoke also exposed load-generator interval drift. Fixed-clock arrivals replaced that generator before the reported measurements. Pending gates: complete current-head CI, final-head Greptile review, and the real run/checkpoint/recovery journey. The full local test run remains explicitly incomplete. The four concrete review findings were fixed. The prior head received 5/5; final-head review remains pending. Local evidence remains outside Git: `/private/tmp/backup-publication-measurement-20261102.json`, `/private/tmp/native-backup-focused-final.log`, `/private/tmp/native-backup-typecheck.log`, `/private/tmp/native-backup-test.log`, `/private/tmp/native-backup-test-interruption.json`, `/private/tmp/native-backup-build.log`, and `/private/tmp/native-backup-harness-check.json`. The harness-check receipt points to its isolated metadata and worker-failure logs. The final client-failure and baseline-control logs are `/private/tmp/backup-benchmark-check-gprcfiiz/failed-client-cleanup.log` and `/private/tmp/backup-benchmark-check-gprcfiiz/final-probe-baseline-control.log`. These local paths are retained for the author; the commands above let reviewers reproduce the synthetic proof. ## Risks - Asynchronous removal adds filesystem-pool work and used more process CPU in the fixture. Check resource pressure as well as responsiveness before rollout. - Filesystem operations can now interleave with unrelated work. The existing session owner still controls publication. Cleanup remains awaited; no deletion is detached and no scheduler is added. - Errors must still fail publication or restore the prior backup as before. Corruption checks and destructive lease authorization are unchanged. - This synthetic macOS fixture does not represent every Linux volume or backup shape. It does not qualify a deployed fix. - Rollout acceptance should confirm the serving SHA, checkpoint completion, continuation/recovery, error rate, and loop delay under matched backup load. Revert if cleanup or recovery fails. ## Model Used OpenAI Codex, GPT-6 family, with tool use and code execution. The exact served model ID and context-window size were not exposed to this agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 17:00:42 +00:00
AlexFrontegg	8424a51018	fix(redaction): scan long letter/digit runs in linear time (#16999) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip removes secrets from run logs, tool results and diagnostics with `redactSensitiveText` before it stores or shows them. > - The JSON field-name patterns in that function start with an unanchored `[A-Za-z0-9_-]*`. Each start position inside a long run of letters and digits scans to the end of the run, so the cost grows with the square of the run length. > - Base64 of text is one long run of letters and digits. A 40 KB base64 blob of source code in a shell command takes about 5 s to redact, and a large Gmail RAW tool result does not finish. > - The redaction runs on the main event loop, so one such chunk blocks every request and every run on the instance. > - This pull request makes these scans start only at the start of a run. The scans become linear, and the redacted output does not change. > - The benefit is that large base64 payloads no longer freeze the server, and every secret that was redacted before is still redacted. ## Linked Issues or Issue Description Fixes #15393. The tool content guard runs `redactSensitiveText` on tool results. A Gmail RAW body is base64url of a MIME message, which is one long letter/digit run. `validateToolContent` on a Gmail-shaped result with a 68 KB `raw` field takes 59 s on master and 19 ms with this change; a 5.4 MB `raw` field takes 0.65 s. Thanks @ts-beat for the clear report in #15393. Refs #12309 (false positives in the same patterns; this pull request does not change which text matches). I searched issues and pull requests for "redaction", "redactSensitiveText", "quadratic", "backtracking", "hangs the server" and "100% CPU". #15393 reports the hang. No open pull request fixes it. **Subsystem affected:** secret redaction (`server/src/redaction.ts`, `packages/adapter-utils/src/command-redaction.ts`), used by run logs, tool content guards and run diagnostics. **Problem:** `compactRunLogChunk` redacts each run log chunk before it truncates the chunk. A shell command such as `printf '%s' '<base64>' | base64 -d > file.ts` with base64 of source text makes one long letter/digit run. Base64 of ASCII text contains `+` or `/` only where every third byte is `>`, `?`, `~` or DEL, so random base64 does not show the problem. One `redactSensitiveText` call on such a tool-call line takes (local measurement): | Blob | Before | After | | --- | --- | --- | | 4 KB | 66 ms | 1.1 ms | | 8 KB | 272 ms | 0.5 ms | | 16 KB | 1.1 s | 1.0 ms | | 40 KB | 5.4 s | 2.5 ms | | 1 MB | not measured | 36 ms | The `\b`-anchored command patterns (`KEY=value`, JWT, `Authorization`) and the CLI option pattern have the same quadratic cost on dash-separated runs, for example base64url data. **Proposed solution:** start the unanchored field-name scans only where no name character precedes them (`(?<![A-Za-z0-9_-])`). A prefixed name such as `myApiKey`, `x_auth_token` or `GITHUB_TOKEN` still matches, because the scan from the start of the run absorbs the prefix. **Alternatives considered:** - Truncate or cap the input before redaction. Rejected: a cut can split a secret and leave part of it unredacted, and the scan stays quadratic below the cap. - Run the scan in a worker thread with a deadline, as #15393 suggests. That limits the damage, but the CPU cost stays, and a timeout must then drop or block the content. It is still useful as an additional defense. ## What Changed - `server/src/redaction.ts`: `JSON_SECRET_FIELD_TEXT_RE` and `ESCAPED_JSON_SECRET_FIELD_TEXT_RE` start a field name only where no name character precedes it. The lookbehind comes after the optional opening quote, so quoted keys behave as before. The `Authorization` word scan uses the same start instead of `\b`. - `packages/adapter-utils/src/command-redaction.ts`: the `KEY=value` pattern uses the same start. The JWT pattern starts only at the first word character of a run. A CLI option name can no longer span `--`, so each `--` starts its own short scan. - Redaction still runs before truncation. `compactRunLogChunk` does not change. Why the output does not change: when an old pattern could match from the middle of a run, it could also match from the start of that run, because the `[A-Za-z0-9_-]*` prefix absorbs the extra characters. The engine tries the start of the run first. So the old patterns never matched from the middle of a run, and the new start condition only skips positions that cannot produce a match. The old JWT pattern also always matched from the first word character of a run. A CLI option name always ends at the end of its run, so the match from the last `--` in the run ends at the same place. ## Screenshots No UI change. ## Verification - New tests: - prefixed secret field names in plain, single-quoted and escaped JSON are still redacted, also when the name is glued to the end of a long run; - a 48 KB base64-of-source tool call through `compactRunLogChunk` stays intact, the secret in the same command is redacted, and the call finishes within 1 s; - `redactSensitiveText` and `redactCommandText` handle 48-120 KB letter/digit, dash-separated, `--`-separated and base64url runs within 1 s, and redact the secret that follows each run; - `x--token`, `--x--token`, `a-b--api-key=`, `prefix-MY_API_KEY=` and `--<jwt>` are still redacted, and `my-token is here` is not changed. - Without the change, the timing tests fail: 9.3 s, 9.8 s, 32 s and 60 s against the 1 s budget. The correctness tests pass with and without the change. - Mutation check: I reverted each of the six pattern changes one at a time. Each revert makes at least one test fail. - Differential fuzzing: 1.1 million random inputs built from secret names, quotes, escaped quotes, dashes, dots, colons and JWT parts gave the same output from the old and new `redactSensitiveText`, `redactCommandText` and `redactDiagnosticText`. - `cd server && npx vitest run <files>`: all 33 server test files that use the redaction helpers, with embedded Postgres, 1730 passed. - `npx vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/command-redaction.test.ts`: 25 passed. - `npx vitest run --project @paperclipai/adapter-utils`, complete suite: 80 of 82 files passed in the parallel run. The 2 other files (`github-launcher-environment`, `sandbox-managed-runtime`) had 5 s timeouts in git-based tests and passed on rerun. ## Risks - Lookbehind assertions need ES2018. `redactCommandText` also runs in the UI bundle. Node and current browsers support lookbehind, and the server (`server/src/log-redaction.ts`) and the UI (`ui/src/components/task-chat/task-chat-attachments.ts`) already use it. - A change to a redaction pattern could leak a secret. The reasoning above, the unchanged existing tests and the differential fuzzing show the same output as before. - Not changed here, found while checking: `redactDiagnosticText` backtracks exponentially on an escaped JSON secret value made of many escaped backslashes with no closing quote, and `redactSensitiveText` redacts a value only up to its first quote of either kind (`{"apiKey":"abc\"tail"}` keeps `tail"`). Both behave the same before and after this pull request. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) by Anthropic, in Claude Code, with tool use (shell, code search, and test runs) and extended thinking. A human directed the design choices. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-02 10:49:08 -06:00
DottaandPaperclip	0748b90062	fix(server): stop goal-less native runs before board task closure (#18616) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task owns its active run and its tool access. > - A board user can close a task while that run is still working. > - The task update route stopped native runs only when they had an active session goal. > - Providers without goals kept running after the route cleared task ownership. > - This pull request stops those native runs before a board closure releases ownership. > - The benefit is an orderly stop instead of tool denials and failed finalization. ## Linked Issues or Issue Description **What happened?** A board user marked a task Done while its native OpenCode run was active. The route cleared its execution lock without stopping the provider. Later tool calls failed with `paperclip_runner_tool_binding_not_authorized`. The final wake fence then failed with `native_current_wake_comments_binding_changed`, leaving workspace finalization in retry. Cancelled also released ownership before its later Stop call. **Expected behavior** A board closure must confirm native Stop before it releases task ownership. A failed Stop must leave the task open. The provider does not need to support session goals. **Steps to reproduce** 1. Start a native task with a provider that has no active session goal. 2. Mark the task Done while its run is active. 3. Observe that the old route updates the task without cancelling the run. ## What Changed - Reuse the existing audited cancellation path before a board user marks an active native task Done or Cancelled. - Keep the existing 409 response when Stop is unconfirmed. - Record the board user in the Stop metadata. The existing acknowledged-Stop policy then leaves queued input and recovery quiet until a later explicit wake. - Inspect additional runs only when a task has an execution owner; tasks without a run lock keep their existing completion path. - Keep native semantic completion, legacy completion, and goal control behavior. - Add route regressions for stop ordering, failed Stop, and legacy completion. - Update the execution contract and task maps. ## Verification - Red before the fix: three regressions failed. Done had no Stop; Cancelled stopped after the update; an unconfirmed Stop allowed Done. - A second red proved that a Stop receipt without the route's operator metadata was not recognized by the real recovery policy. Both cases pass after the metadata repair. - Final local result on macOS arm64 / Node.js 26.4.0: **348 tests pass across eight suites**. These cover the route, native self-completion authority, cancellation, acknowledged Stop, recovery queues, dependency wakeups, and completion telemetry. - CI found seven sibling route failures from an unnecessary active-run lookup on tasks without execution ownership. Those cases failed locally before the correction and pass afterward. - Earlier concurrent local runs hit a module-load hook timeout and one watchdog socket hang-up. The isolated watchdog case and the complete final eight-suite run pass with a 90-second startup budget. No assertion was removed. - `node scripts/check-maps.mjs` and `git diff --check`: pass. - `pnpm build` and `pnpm -r typecheck`: pass. Server build and typecheck also pass after the Stop metadata change; the final server build includes the lookup correction. - `pnpm test:run`: attempted; the unchanged chat shard selector failed before executing its selected tests (`No test found ... in lines ...`). This is separate from the PostgreSQL startup failure. CI supplies full-suite verification. - PostgreSQL integration and live-provider acceptance did not run locally. Embedded PostgreSQL failed before app startup with `could not create semaphores: No space left on device`. No usable isolated local database runtime was available. CI supplies the PostgreSQL regression environment. The full live OpenRouter workflow remains unverified. - The first final-head CI pass hit a 15-second timeout in the unchanged `attention-retention-work.test.ts` blocker-history fixture. Cleanup then deadlocked and later inserts reused IDs. This service test does not call the changed route. The original log is [retained here](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289220). The one allowed rerun also failed on the same commit: the `running` case exceeded 15 seconds again ([rerun log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125393915025)). The rerun otherwise passed 2,346 tests. No retention source or assertion was changed. This repeatable failure is the remaining CI gate; the PR is **not ready to merge**. The next step is to isolate that retention fixture/query in a usable PostgreSQL environment and repair its cause before rerunning CI. - Greptile reviewed final commit `b3cb41af2d911e9ced219e846cada3ecec835dbf` at **5/5**, with no unresolved review threads. Greptile is green. CI is blocked by the repeated retention test timeout described above. - CI native cancellation evidence: all 16 tests in `server/src/__tests__/native-cancellation-request.integration.test.ts` pass ([job log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289222)). - CI PostgreSQL wake-queue evidence: all 96 tests in `server/src/modules/wake-queue/adapters/postgres.test.ts` pass ([job log](https://github.com/paperclipai/paperclip/actions/runs/41476191754/job/125384289470)). ## Risks Board closure now waits for native Stop even when the provider has no goal. A failed or uncertain Stop keeps the task open. The change uses the existing cancellation and workspace-preservation path. It adds no retry loop, credential access, schema change, or publication authority. Direct service updates and agent semantic completion retain their existing paths. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, shell execution, and browser inspection. The exact serving snapshot ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 10:41:44 -06:00
DottaandPaperclip	912cff9392	fix(ui): link chat message identities to agent overviews (#18615) ... ## Thinking Path > - Paperclip lets people manage and converse with agents. > - Chat replies identify their author with an avatar and name. > - Those identities currently offer no path to the responding agent's profile. > - The existing overview route already provides that context within the current company. > - This PR links the shared chat identity to that overview while preserving its layout and typography. > - People can inspect the responding agent by clicking either its name or avatar. ## Linked Issues or Issue Description Related: #17169 links the conversation's page header. This change covers the author identity above messages and active turns. **What existing behavior does this improve?** Navigation from chat message authors to their agent overview. **Subsystem affected** ui/ — shared task and standing-chat identity component. **Current behavior** An agent's name and avatar above its message are plain text and an image. **Proposed behavior** The name and avatar form one company-scoped link to the actual author's overview. Both retain their appearance, including no underline on hover. An identity without an agent ID remains plain text. **Reason and benefit** Readers can inspect the responding agent directly from the conversation. **Breaking changes** None; this adds navigation to an existing read surface. ## What Changed - Link the shared agent identity to its existing overview route when an ID is available. - Preserve the existing avatar, text, spacing, and non-underlined appearance. - Document this entry point and the desktop/mobile verification recipe. ## Verification - `pnpm -r typecheck` — passed. - `pnpm build` — passed. - Existing bubble, runner-turn, turn, task-thread, and AgentChat tests — 461 passed. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `git diff --check` — passed. - Chromium production `TaskChatThread` in `Layout` (mixed-author Storybook fixture): dark and light desktop identity screenshots are pixel-identical before/after and on hover; no underline, and the href points to the responding author in the same company. The preview is UI evidence, not backend/provider acceptance. The separate standing-chat preview had stale history/email fixtures, and the optional mobile preview placed the identity behind its composer; neither is claimed as acceptance evidence. - Added router integration tests click both name and avatar from a different agent's chat and assert the author's company-scoped overview; unknown IDs remain unlinked. All 30 bubble/identity tests pass. - Full UI project: 10,284 tests passed, one 5-second timeout in the unchanged design-sandbox snapshot test. Its isolated rerun passed all 12 tests; the initial timeout remains recorded, and its cause was not established. - Final UI typecheck passed after adding the tests. - `pnpm test:run` stopped before running tests: its unchanged server chat shard filters selected no tests. The full UI result and isolated rerun are recorded above. This is not a claim that the full local repository suite passed. - [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/41475021544) passed, including build, typecheck, unit/integration suites, all browser shards, and both aggregate gates. Greptile is 5/5 with its review thread resolved. The separate Paperclip Review check is still queued, waiting for its assigned agent; no assessment has been published yet. ## Risks Low risk: navigation uses the existing company-aware router and explicit author ID. Unknown identities stay unlinked. The shared identity is used by standing chats, task replies, and runner headers. ## Model Used OpenAI GPT-6, running as Codex with reasoning, code editing, and tool execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 10:41:00 -06:00
Dotta	a6afe3e230	Fix TestOps launch preflight search encoding (#18620) ... ## Thinking Path > - Paperclip manages AI agents that do real work. > - Product evals verify the saved outcome of that work. > - TestOps plans select and dispatch these evals. > - Operator launchers check a unique launch name before they request work. > - TestOps expects an encoded filter, but the launchers send plain name text. > - This change uses the vendor filter format in the shared TestOps API owner. > - The existing checks can then prevent duplicate launch requests. ## Linked Issues or Issue Description Refs #18516, #18608, and #18542. The completion pilot registered successfully after #18608. Its optional API launcher then stopped with HTTP 400 in [run 41477551241](https://github.com/paperclipai/paperclip/actions/runs/41477551241). It stopped before the launch mutation marker. No evaluation was dispatched by that request. The launch search API uses Base64 of UTF-8 JSON filters. The launchers sent plain name text. I confirmed the format in the deployed public [vendor filter codec](https://paperclip.testops.cloud/es-testops-CICoDnkU.js) and its launch UI. The encoded name filter displayed the existing pilot launch. The TestOps UI separately dispatched [the genuine three-case pilot](https://github.com/paperclipai/paperclip/actions/runs/41478906153); this PR does not create another run or claim that pilot passed. ## What Changed - Add the typed name-filter encoder to the existing TestOps API module. - Use it in the backlog and completion launch preflight requests. The full-catalog route uses the same completion adapter. - Preserve the exact launch name, nonce, selection, authorization, once-only markers, and no-resend behavior. - Add protocol controls through both real launch functions. Document the wire format. ## Verification - Red: both new consumer controls reproduce the HTTP 400 boundary with plain name text. The other 109 API controls pass. - Green: all 111 API controls pass after the fix. All 166 related API, pilot contract, and full-catalog controls pass with Node 24.22.2. - The first broad local invocation lacked the `yaml` dependency in the new worktree. Its failure log is retained. Reusing the existing installed dependencies allowed all 166 controls to run and pass without further source changes. - The vendor-format query displayed the existing uniquely named pilot launch in the authenticated TestOps UI. No launch POST or model call was used for this read-only verification. - Diff check passes. All required CI checks pass on exact head `b55bdc6753fa042e4c2465e7b9b366b8126b4730` ([run 41480643021](https://github.com/paperclipai/paperclip/actions/runs/41480643021)). Both current-head reviews are 5/5, with zero unresolved findings. No TypeScript, compiled product, workflow, grader, model, clock, retry, or evidence-boundary changes. ## Risks The vendor can change its search format. The format is kept in the existing provider-specific API module, and both launch consumers have protocol controls. Existing duplicate detection and uncertain-response refusal remain in place. The already-running UI pilot is independent of this fix. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, code execution, and native accessibility tool use. The host does not expose the exact model revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-02 10:40:23 -06:00
Nicky LeachandPaperclip	f8a888a072	feat(runner): define bounded startup lifetime contracts (#18490) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Native sessions and plugin workers have separate startup lifetimes. > - A caller timeout does not prove that a worker or an accepted host write has stopped. > - Shared startup code needs an exact deadline and an actual settlement contract. > - This PR adds those contracts and tests them before any production caller uses them. > - The dependent integration in #18405 will use the contracts with verified cleanup. > - This split lets reviewers assess the lifetime rules separately from runtime activation. ## Linked Issues or Issue Description Refs #18405. This PR extracts the foundation from that change. The integration will depend on this PR. **Problem or motivation** A bounded startup must distinguish the caller's wait from actual worker completion. A late worker reply or an accepted host request can outlive the original RPC promise. A shorter timeout must not discard that responsibility or authorize a replacement run. **Proposed solution** Add one Core execution context, optional Runner startup deadlines, and worker settlement and retirement APIs. Test the contracts with real child-process IPC. Keep production context creation, transport projection, capability negotiation, durable cleanup, and Stop/recovery integration in #18405. **Alternatives considered** One combined PR mixed the lifetime contract with its activation and cleanup policy. Activating a shorter deadline before the complete transport and cleanup path is unsafe. Splitting only the SDK declarations would not provide a useful behavioral review boundary. **Roadmap alignment** This supports the existing plugin, sandbox execution, and bounded recovery capabilities. It adds no new product workflow, endpoint, migration, provider or runtime dependency. ## What Changed - Add `NativeExecutionContext`. It retains the selected input, binds one transport target, rejects work after cancellation or expiry, and tracks actual worker settlement. Expiry alone does not complete pending work. - Add worker startup groups. A caller timeout retains the group's responsibility. Retirement revokes obsolete invocation scopes, waits for accepted host handlers, and retires only the verified group's bookkeeping. Each cleanup wait is bounded; an unfinished handler remains pending for a later retry. - Add an optional Runner startup signal and absolute deadline for session open and retained attachment. Clean up abort listeners after synchronous failures. Existing callers keep their default timeout path. - Expose the remaining confirmed controller-lease budget without a new query or renewal timer. - Add optional SDK context and preparation-retirement fields. No production bridge sends or advertises them in this slice. - Keep the required lease mock update with the new `remainingMs()` contract so this foundation typechecks independently. Production activation files, including the native executor and environment task bridge, match the current master base. No production caller constructs `NativeExecutionContext` or passes the new Runner startup option. This PR adds no durable cleanup producer and claims **no standalone TTFT improvement**. The dependent integration must enable the context, transport and verified cleanup together. ## Verification Initial split verification on `8ebc8a2f9c6c02e90f920356138b3742a4067b28`, based on `9a3f395737e7a92faf99971459979061d423531a`: - From `server`: `../node_modules/.bin/vitest run src/services/native-runtime/native-execution-context.test.ts src/services/native-runtime/native-execution-context-worker.test.ts src/services/native-runtime/native-execution-lease.test.ts` — **17 passed**. Includes real child IPC, caller timeout with late reply, worker death, accepted host-handler draining, obsolete cleanup invocation rejection, unrelated work preservation, context scope, expiry, and confirmed lease budget. - From `packages/paperclip-runner`: `../../node_modules/.bin/vitest run src/native-session-runtime.test.ts` — **188 passed**. Includes existing callers without the new option and new synchronous-failure/expired-deadline cases for fresh and retained sessions. - Direct TypeScript checks for `packages/plugins/sdk/tsconfig.json` and `packages/paperclip-runner/tsconfig.json` — passed. Built only their shared/eval TypeScript prerequisites. - From `server`: `../node_modules/.bin/tsc --noEmit` — passed. - Checked the partition against the reviewed combined candidate. The twelve selected files match exactly. The thirteenth file carries only the required test-mock update. The production executor, bridge and target wiring remain equal to the base. - `git diff --check`, added-line secret/private-data scan, and added-code-comment scan — passed. No generated files or run artifacts are committed. - The first old-base split passed its focused tests but failed server typechecking because the unchanged executor test mock lacked `remainingMs()`. The original failure is retained. Moving that one-line mock update into this foundation resolved the split dependency; the final current-master checks above pass. - Full local CI suites, Rust builds, deployments and provider calls were not run, as requested. GitHub CI will provide full regression coverage. Current-head GitHub CI, standard review and Apex are complete; exact acceptance is recorded below. Review follow-up `d7635f4d280dab757a3666703737bcda955c9361` fixes three reproduced lifetime edge cases. A retiring startup group rejects new calls and shares one retirement promise; failed bounded retirement remains retryable. Transport identity uses a fixed field tuple rather than object-property order. Finishing a context rejects pending lookup waits without aborting an already-started backend or pretending the worker completed. All three new regressions failed before the fix; the updated foundation selection passes **20 tests**, including actual IPC and failed-retirement retry. Server typechecking and added-line checks pass. These replace the earlier 17-case selection, rather than adding 20 unique cases to it. The original failures and passing reruns are retained. Apex follow-up `cb8860bab5ab9d6357a352e11f276be0d0cdb26d` keeps removed worker handles reachable while they own startup calls, accepted host handlers or retirement work. Drain/retirement includes active and retained handles; completion callbacks remove settled handles. Stop and replacement remove only the exact handle they observed. This adds no database/network operation, polling loop or generic shutdown wait. The SDK, Runner, context and transport bridge are unchanged. Seven additional IPC regressions failed on the prior manager and pass after the fix, covering removal/replacement, Stop-all, host-handler settlement, bounded retirement failure/retry, pruning, and replacement ownership. The updated foundation selection passes **27 tests**, with server typechecking passing; these supersede the earlier 20-case selection. A separate real-PostgreSQL/child-IPC characterization uses the integration's actual SDK host, storage registry and run-control path with the exact corrected manager bytes. Before the fix, cleanup released run ownership while an accepted host write was blocked, then the late write restored obsolete metadata. After the fix, ownership and cleanup remain pending until the write completes; cleanup then removes the stale metadata before releasing ownership. The original failure, corrected result and source snapshots are preserved. Final head `cb8860bab5ab9d6357a352e11f276be0d0cdb26d` passes GitHub CI (53 successful checks, 2 expected skips, Snyk success), standard Greptile **5/5** and fresh Apex **5/5**. All review threads are resolved and the PR is conflict-free. ## Risks The shared Runner helper now observes synchronous operation failures through its asynchronous cleanup path. The affected runtime file's 188 tests pass, but full regression coverage still depends on CI. The new lifetime APIs require honest settlement signals. A future caller must not treat a deadline, worker crash or Stop intent as verified retirement. Production activation remains dependent on the complete integration and its cleanup tests in #18405. This foundation introduces no shorter production startup budget or new cleanup policy by itself. The context's transport projection is process-local. It must not be persisted or used as a general authorization cache. No production caller supplies it in this PR. There is no database migration, new dependency, or deployment requirement for merging this foundation. ## Model Used OpenAI Codex, based on GPT-6, assisted with repository inspection, the split, tests and command execution. The exact serving model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal issue identifier - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — this description documents the inactive contracts; operational activation documentation remains with #18405 - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on `cb8860bab5` - [x] Current-head standard Greptile and Apex are 5/5 with no unresolved findings - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 08:22:42 -08:00
DottaandPaperclip	412834a860	test: overlap long recovery and maintenance waits (#18544) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - CI verifies recovery, execution authority, and background work before changes merge. > - Diff selection saves work, but a selected file still runs its cases in sequence. > - The three longest files in a recent full master run each took over eight minutes. > - Many of these cases must observe real lease expiry or idle periods. > - This pull request gives independent cases separate fixtures so those periods can overlap. > - The benefit is a shorter test path with the same assertions and observation windows. ## Linked Issues or Issue Description Refs #18464. The latest measured successful master [Cloud readiness run](https://github.com/paperclipai/paperclip/actions/runs/41300933157) spent 581 seconds in issue recovery, 569 seconds in native restart recovery, and 512 seconds in maintenance server routes. File-level selection cannot divide those long serial files. Shortening their lease or idle windows would weaken the regressions they protect. ## What Changed - Split issue recovery into four files with separate database and worker fixtures. Keep all 18 cases. - Split native restart coverage into recovery, lease renewal, preparation, checkpoint cost, and disk-loss files. Keep all original 79 cases, including the existing container-qualified cases. The master update also retains its newly added recovery cases. - Close each native case's database clients after pending failure reports drain. The shortened file must not rely on idle timeouts to stay below PostgreSQL's connection cap. - Run the native Runner lane with two isolated fork workers. Register every new file in the lane and verify that the partition still includes every suite. - Give each maintenance case an owned database. Run at most two cases together with test-local assertion contexts. Keep both real 90-second quiet windows and all 17 cases. - Preserve serial execution when any explicit shared `PAPERCLIP_*_TEST_DATABASE_URL` is present. An admin URL that creates a separate database per fixture can run concurrently. - Refresh shard weights from the protected master run. Mark the initial split-file weights as conservative estimates until master measures them. - Anchor the attention-retention fake clock to PostgreSQL time. Its fixed clock started trailing real database timestamps after 12:00 UTC on November 1, causing an archive-deadline test to time out. Keep its deadline assertions and timeout unchanged. - Document the fixture and concurrency rules. Leave runner authorization and workflows unchanged. ## Verification - Final head `c9a4ff11b3ba81277341294c7721b34fb8705ede`: [full PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/41477848686), including typecheck, build, all test shards, E2E, and the final verification gate. The rollup has 55 successful checks, one neutral review check, four expected skips, and no pending or failed checks. Greptile reviewed this exact head at 5/5 with no actionable findings. - November 2 master update: merge commit `c9a4ff11b3ba81277341294c7721b34fb8705ede` resolves the native recovery import conflict while retaining the new busy-row and remote continuity cases from master. All 38 test-runner checks and the three new busy-row cases passed locally. - Retention verification after the master update: the first local full run passed 52 cases and timed out at 15 seconds in the paged-backlog case. That case passed in isolation. The unchanged master suite passed all 53 cases (82.95 seconds), and a full PR rerun also passed all 53 (76.65 seconds). The initial timeout cause remains unproven; no timeout or assertion was weakened. - Previously verified head `fd288a8e8c88df54dcdca7b7e42366b019182782`: [full PR CI passed](https://github.com/paperclipai/paperclip/actions/runs/41312625439). There are 55 successful checks, one neutral review check, four expected skips, and no pending or failed checks. Greptile reviewed this head at 5/5 with no actionable findings. - Issue recovery: 18/18 passed before and after the split on the same local machine and PostgreSQL fixture path. Wall time changed from **630.6 seconds to 319.7 seconds** (49% less). All case titles match. - Maintenance also passed all 17 cases on AWS: structured file cost dropped from **511.8 to 282.6 seconds** across the linked master run and the first PR run. These are different CI runs, so the matched local comparison below is the stronger isolated measurement. - Maintenance: 17/17 passed before and after with the restored pinned dependencies. Wall time changed from **488.1 seconds to 317.2 seconds** (35% less). All 120 assertion lines remain. Both real 90-second windows also passed a final targeted run. - Native restart: 71 passed and eight existing container-only skips before and after the split. Wall time changed from **632.1 seconds to 346.8 seconds** (45% less). All 79 cases and 254 assertion lines remain. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - Extra TypeScript checking of the moved recovery and maintenance tests passed. The standard server config excludes `src/__tests__`. A separate check of attention-retention fixtures reports existing type errors; the unchanged baseline has identical diagnostics, with none introduced by the clock fix. - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs scripts/ci/test-selection*.test.mjs`: 61 passed. - `pnpm test:run`: stopped in the second chat partition: 335 passed and four failed (two GitHub retry-state assertions and two Teams timeouts), after 339 passed in the first partition. These chat tests are unchanged. All four pass in a focused run (19.1 seconds). The full-suite ordering/load cause remains unproven. All three CI chat jobs passed. The previously failing cases passed there in 1.3, 1.3, 2.4, and 2.2 seconds; see [the chat job log](https://github.com/paperclipai/paperclip/actions/runs/41309691650/job/124857580714). The attention-retention timeout repeated in CI and in an isolated local run. Its fake clock was fixed at November 1 noon, which had fallen behind real PostgreSQL timestamps. Anchoring it to the database clock passes all 53 cases; the failed case now takes 425 ms. - The native CI lane passed all 162 runnable cases with eight existing skips after client cleanup: **650.7 seconds on the earlier master run to 391.0 seconds on this PR**. - CI found native connection exhaustion after the long waits moved out of the main file. The same failure reproduced locally at a 100-connection cap. Per-case client cleanup fixes the shortened file at that cap, and all five native files now pass together at that cap (71 passed, eight existing skips). - Early attempts exposed a missed fixture reference and a missed reporting setup hook during extraction; both were restored. The first CI policy run also required moving the new native fixture under test helpers; that boundary check now passes locally. A separate auxiliary command picked up the host pnpm version and removed dependencies. The original maintenance baseline reproduced the same missing-package failures, so that failed attempt is excluded from timing comparisons. A later unchanged-baseline run passed all 17 cases. The pinned install was restored before the passing maintenance run. The audit also covered heartbeat process recovery, deferred receipts, chat, workspace recovery, attention, and connection management. Their shared fake clocks/provider mocks or existing partitioning need different changes. The 16,400-row attention case and the large checkpoint stream remain intact because their size protects real limits. ## Risks - Two isolated processes use more memory and database connections than one. Concurrency is capped at two. Shared database overrides force serial execution. - Moving setup into helpers can change test lifecycle order. The split suites keep the same case assertions and use independent process state; focused runs and the full CI suite check the move. - Split-file weights are estimates based on case timings from the linked run plus repeated setup cost. The next protected-master refresh replaces them. Weights never permit tests to be skipped. - Local timing varies with host load. CI measurements after merge are needed to establish the overall PR wall-time improvement. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, tool use, and code execution. The exact backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (changed suites and focused reruns; the full local-run caveat is recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 10:19:24 -06:00
DottaandPaperclip	baa4a2e64e	fix(runner): restore Codex auth before source recovery (#18612) ... Restore disposable Codex authentication and safe provider configuration before cold source recovery. Preserve source receipts and conversation state until reconciliation, and rebuild assigned tools from the admitted launch binding. Validate remote launch destinations, clear stale authentication when no selected credential exists, and atomically publish SSH file transfers without following retained links. Red/green recovery regressions, a real two-turn Codex/MCP probe, and all current PR checks pass. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-11-02 10:14:24 -06:00
DottaandPaperclip	42c64d4ec4	fix(runner): preserve deliberate OpenCode approval waits (#18610) ... ## Thinking Path > - Paperclip coordinates agent work and human approval. > - A saved Plan must wait for the user before work continues. > - The native runner interrupts the provider after it saves that wait. > - OpenCode can report the interruption with an idle event and no final answer. > - Empty-answer recovery treated that intentional pause as an incomplete answer. > - This change stops answer recovery for an interrupted turn. > - The runner still requires a real provider terminal before it settles the wait. ## Linked Issues or Issue Description **What happened?** OpenCode could send a corrective model request after Paperclip interrupted a turn for a saved approval request. The native runner then failed with `governed_wait_provider_terminal_unproven` because it did not receive the required terminal event within the existing drain window. **Expected behavior** An interrupted turn must stop requesting answers. A provider idle or cancelled event must remain available to the native wait settlement. An HTTP abort acknowledgement alone must not count as a terminal. **Steps to reproduce** 1. Start an OpenCode turn that has no final answer. 2. Interrupt that turn after saving a durable approval request. 3. Have the provider return `session.idle` or an idle `session.status`. 4. Observe that the old driver sends a second prompt instead of emitting the terminal event. The controlled regression fails before the fix for both idle event forms: it observes two prompts instead of one. The affected driver blob is identical in the deployed source and the current base. Related: #18364 added the empty-answer recovery path. ## What Changed - Record interruption intent for the active OpenCode turn before asynchronous settlement. - Cancel its answer-recovery timer and prevent corrective requests for that turn. - Preserve provider terminal checks and avoid a delayed abort of a later turn. - Cover both idle event forms, an existing correction, durable question handoff, in-flight tools, missing terminal evidence, and a later authorized turn. - Document the interruption contract. ## Verification - The new regression failed before the fix for the expected extra prompt. A second red test confirmed the sibling durable-question handoff also sent that unwanted prompt. - Interrupted idle events now produce explicit cancellation terminals. Tests reject a failed terminal and require normal completion on the later authorized turn. - All 305 targeted tests passed across the OpenCode driver, normal empty-answer recovery, interruption, backend publication, and native session runtime suites. - Runner TypeScript check passed: `tsc -p packages/paperclip-runner/tsconfig.json --noEmit`. - `git diff --check` passed. - Greptile reviewed the current commit at 5/5 with no findings. Both earlier Paperclip review findings are fixed and their threads are resolved. - The additional Paperclip advisory reviewer could not assess the final commit because its earlier workspace save did not finish. Its check is neutral; this is not a passing assessment. No reviewer configuration or workspace safety gate was changed. - Repository CI passed on run 41470761437, attempt 2. Attempt 1 retained a 15-second timeout in an unchanged voice-report test; only the failed jobs were retried once, on the same commit. The identical test also passed in an isolated baseline check against disposable PostgreSQL with its original timeout. - Live QA acceptance is pending. No eval prompt, model, allowance, timeout, or assertion changed. ## Risks The change affects OpenCode interruption and answer recovery. A silent provider must remain unproven; the negative control checks that case. Ordinary answer recovery remains enabled on authorized turns. No schema, permission, or provider configuration changes are required. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The runtime does not expose a more specific model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> canary/v2026.1102.0-canary.2	2026-11-02 10:07:32 -06:00