mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-30 13:20:54 +01:00
2aa32647bfd3fdb331cd921e7a6a412f035c0bca
263
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
2aa32647bf |
docs: record retention review repair and remaining cursor audits
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
a29ce265b0 |
fix(server): preserve database precision in retention recovery cursors
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
2f334c678b |
docs: record approval retention local verification
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
ea3dd40cdd |
fix(server): schedule approval retention from durable source deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
f240381fd8 |
docs: record live review and CI evidence for sleep stack
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
b20a7f6741 |
docs: audit attention retention sources before scheduling migration
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
706beb01f2 |
test(server): verify Grok dispatch lifecycle and quiet history
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
1bbfc69590 |
fix(server): retain full target scans with scoped transaction hints
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
5062efb368 |
fix(server): invalidate decision targets from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
e7db8ce543 |
docs: audit decision target invalidation producers
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
a5b5994c1d |
fix(server): recover decision effects from saved claim deadlines
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
67bf465e3b |
docs: record decision sleep PRs and effect recovery audit
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
63b58cc26f |
test(server): bound and qualify decision expiry lifecycle
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
5f96892a58 |
fix(server): serialize decision outcomes and bound continuation shutdown
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
66bbef521e |
fix(server): wake decision continuations from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
7ea1fbd8e1 |
docs: record eighth sleep PR and decision continuation audit
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
692e3ee02c |
fix(server): wake archive notification delivery from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
0956883a84 |
fix(server): wake tool-review card repair from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
db2420ef1a |
docs: record sixth sleep PR and projection follow-up
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
85003a7ac4 |
docs: record interrupted-tool recovery contracts and verification
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
7a1cf03e62 |
test(server): require settled outcomes for approved Test-tab calls
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
1cdb26a68b |
fix(server): separate tool outcome persistence from provider errors
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
5dcd193d35 |
fix(server): recover approved tool actions from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
38c4e28135 |
docs: audit approved tool recovery ownership for next sleep slice
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
ce6a260a83 |
test(server): qualify pending review expiry lifecycle
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
77d6a3bdc2 |
docs: record stack evidence and separate tool-review obligations
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
75332b9641 |
fix(server): schedule gateway credential expiry from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
fac808eabc |
fix(server): drive secret-proposal expiry from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
a592550336 |
fix(server): schedule custom-image setup expiry from committed changes
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
71c04c0bfb |
fix: stop idle environment cleanup polling (#17145)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent environments can leave pending teardown or unverified provider resources. > - Recovery must preserve those resources until ownership and workspace safety allow cleanup. > - Empty maintenance scans keep an idle instance and its database active. > - The existing rows, transaction signals, and scheduler can represent the actual recovery obligation. > - This pull request schedules pending cleanup and historical verification only while work remains. > - An empty or fully verified instance stops these scans without losing cleanup recovery. ## Linked Issues or Issue Description **What happened?** The server scanned pending environment cleanup on its maintenance tick. The run reaper also scanned it even when it recovered no runs or active leases. Historical resource verification scanned every five minutes, including when all history already had valid cleanup proof. **What did you expect?** Writes should wake cleanup. Pending rows, deferred orphan buffers, and unverified history should retain recovery deadlines. When none remain, these workers should make no periodic database or provider calls. **Steps to reproduce** Start a configured instance with no pending leases and fully verified history. Leave it idle. Observe repeated pending-cleanup and historical-verification reads from the maintenance paths. Related: #17106 supplies the preceding login-cleanup migration. #13523 improves lease-reaper query indexes. #16809 addresses reusable-sandbox recovery authority. This PR keeps those authority checks in their existing services. ## What Changed - Register two workers with the existing delivery coordinator and shared scheduler. Remove their unconditional maintenance-tick calls. Run the reaper's inline pending cleanup only after it recovers actual work. - Add commit-aware signals to the audited lease and custom-image writers. Existing transactions notify directly; other writers use one small transaction helper. No tables or migrations are added. - Derive pending cleanup deadlines from the existing row backoff and retry or claim metadata. Retain a historical retry only while an unverified candidate exists. - Register unwritten orphan buffers across runtimes that share the database handle. Buffer registration wakes recovery without a database connection. Load the existing spool at startup and retry failed or capacity-limited loads while needed. - Isolate spool failures per runtime. Attempt all buffers and persisted pending rows, then retain a retry for any failed spool. - Document producer coverage, idle behavior, and remaining blockers. Update the sleep inventory. - Repair the inherited CSS-recovery E2E test: the current build no longer emits task-page CSS. Exercise the actual lazy environment stylesheet, preserving real network failure, recovery, and bounded reload assertions. ## Verification October 24 integration update: merged current master and resolved the runtime/spool conflicts. Preserve serialized ingestion of records appended after startup. Plugin replacement failures now register deferred recovery with an idle hold; concurrent callers share one persistence attempt. The producer audit includes this new writer. - Focused cleanup selection: 270 tests now pass across five suites. The initial run lacked a built plugin SDK and executed no tests; building the SDK fixed setup. The new concurrent-flush assertion then reproduced duplicate rows; a single-flight guard fixed it and all 18 plugin integration tests passed on rerun. - Server typecheck passed. After review fixes, all 211 tests across the cleanup worker, real orphan spool, environment runtime, and plugin integration suites passed. These overlap the earlier focused selection. The rate-limit test initially expected a write after twenty idle minutes to wait; corrected it to assert that subsequent writes cannot accelerate the five-minute cadence. - Review fixes preserve retries for unreadable spool directories/records, keep historical provider checks off the startup wait while joining them at shutdown, and restore their independent five-minute minimum spacing. The complete [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/40108905033) passed on final commit `79ae1589c3c7b924db4bedb337d0c7ed3d95020b`, including all E2E shards, server/workspace tests, runner checks, typecheck, build/public-install verification, and security checks. - CI found a stale E2E asset matcher: `issue-detail-*.css` is no longer emitted. Both corrected browser tests passed against a fresh production UI build and disposable local instance. Fresh Greptile review is 5/5 on that same final commit, with all review threads resolved. Previous implementation verification: - `pnpm -r typecheck` and `pnpm build` passed. - New PostgreSQL worker suite: 12 tests passed. Covers quiet configured state, verified history, writes, outer commit and rollback, retry deadlines, restart, drain, deferred orphan buffers, spool failure, pagination, provider uncertainty, shutdown, and the idle run-reaper path. - Worker and environment runtime suites after review fix: 183 tests passed. A new assertion reproduced buffer starvation before the fix. Updated transaction fault injection and isolated simulated-instance database handles. Server typecheck also passed after the fix. - Run-recovery suites: 58 tests passed. Pending-cleanup suite: 38 tests passed after updating the logging fixture to seed real orphan recovery; an idle reaper no longer invokes cleanup. - Stop and workspace-recovery integration selection: 268 tests passed; two live-provider tests skipped by their environment gate. This overlaps the worker suite and is not an additional unique-test count. - Started the full local `pnpm test:run`, then stopped that older duplicate run after CI found the fixture mismatch. The corrected 38-test suite passed locally. The full sharded [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39678174479) passed on final commit `fc0ff3c4554ca4190a8b0b81ec06ef7065b8ad2c`, including build, typecheck, runner, server, workspace, and E2E checks. This is not a claim that the local full run completed. ## Risks - Missing a future producer signal could delay discovery until restart. The producer audit is documented in `doc/idle-environment-cleanup.md`. Independent SQL writers and another server process are outside the existing process-local notification contract. - Historical candidates blocked by ownership or provider uncertainty keep a five-minute retry until they can be verified. Provider destruction, claim fencing, Stop retention, and workspace safety remain controlled by their existing services. - This does not remove orphan-active discovery or legacy shutdown settlement from run recovery. Active custom-image expiry is also still periodic. External wake coverage does not change. - Local tests prove these workers become quiet and recover. They do not establish deployed instance suspension or fleet cost savings. ## Model Used OpenAI Codex (GPT-6), with code execution and reasoning. The exact serving model ID and context size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
341b622604 |
feat(ui): Artifacts V1 behind an experimental setting — Artifacts page, panel tab and task thread (#17228)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents produce artifacts: documents, files, pictures, video and links. The app shows them on the Artifacts page, in the right panel's Artifacts tab and in the task thread > - The artifact directions lab (#17213) settled a presentation for those surfaces, MVP V1, drawn from the PCLP-Core mocks. It is a reference, not product code > - Shipping it means product components on product data, token-only, behind a switch, one surface at a time, so the current presentation stays the default until the new one has been used > - This pull request adds the switch (Settings > Experimental > Artifacts V1), the `--artifact-*` tokens, the V1 primitives as product components, and the V1 presentation on all three surfaces behind the switch: the Artifacts page, the panel's Artifacts tab and the task thread > - The benefit is the redesign in the product, reversible by one setting, with every existing behaviour of those surfaces kept ## Linked Issues or Issue Description No public issue exists. This section follows the feature request template; the plan is `doc/plans/2026-10-22-artifacts-mvp-v1.md`. **Subsystem affected** The web UI (`ui/`): the Artifacts page, the right panel's Artifacts tab and the task thread. The experimental settings contract in `packages/shared` and `server` gains one boolean. **Problem or motivation** The three artifact surfaces grew separately and their artifact presentation is dated and inconsistent. The lab produced a measured direction; nothing in the product uses it yet. **Proposed solution** An experimental setting, `enableArtifactsV1`, default off, with the same plumbing as the other experimental settings. Product primitives under `ui/src/components/artifacts/v1` (kind helpers and adapters, the sheet and print artwork, the stack, the page tiles with the Paper Mono pill, the row the panel and thread will share), driven by `--artifact-*` tokens in `index.css` with light and dark values. With the setting on: the Artifacts page shows one row of views (Tasks, All files, Images, Videos, Documents, Text, Files) and the V1 tiles, a task with several artifacts as a stack whose newest artifact is the first page, everything else as single tiles; the panel's Artifacts tab is one flat list of V1 rows, newest first, with the expandable rows (Markdown review surface, document annotations, deep links) keeping their behaviour behind the row as their face; the thread shows a message's files as rows under its footer and registered deliverables and documents as one row each. The hover movement respects reduced motion. **Alternatives considered** Replacing the current cards outright: no fallback and three surfaces to land at once. Mounting the lab's components: they are typed on lab fixtures and keep lab-local tokens, which the token gates reject. The primitives were ported instead and the lab stays a reference. **Roadmap alignment** Supports the "Artifacts & Work Products" theme in `ROADMAP.md`. Presentation only; no data model change. **Additional context** No new API fields: the tiles show title, identifier, count and "Updated …"; rows take sizes from the work-product and attachment payloads; a picture's aspect ratio comes from the loaded image, 4:3 until it loads. The plan records what each surface keeps and what it drops (run sections in the panel; the "Images · N / Files · N" labels and the four-up grid in the thread). ## What Changed - Add the `enableArtifactsV1` experimental setting: shared type, validator and feature catalog, server defaults and managed keys, the `useArtifactsV1Enabled` hook, and the Settings > Experimental toggle (in its alphabetical place). - Add the `--artifact-*` tokens and the Artifacts V1 component rules to `ui/src/index.css`: light values on `:root`, dark overrides, and the hover movement under `prefers-reduced-motion: no-preference`. - Add `ui/src/components/artifacts/v1/`: `kinds.ts` (kind treatment, labels, glyphs, the server's media-kind rule, adapters from `CompanyArtifact`), `geometry.ts` (the measured constants), `when.ts`, `ArtifactSheet`, `ArtifactPrint`, `ArtifactStack`, `ArtifactFront`, `ArtifactTile` (stack and single tiles, caption, grid), `ArtifactRow` (the row and its inline group), `ArtifactsV1Filters`, with the mock's SVG assets. - Wire the Artifacts page: with the setting on, the toolbar's filter row becomes the V1 views and the grid renders V1 tiles; the toolbar takes an optional `filters` row for this. The old cards and kind filters stay the default. - Wire the panel's Artifacts tab (`IssuePropertiesArtifactsTab`): with the setting on, one flat list of V1 rows, newest first; `DocumentRow` and `MarkdownWorkProductRow` take a `face` render prop so the V1 row fronts their unchanged expansion, review and deep-link behaviour; `PanelFileRow` opens attachments and attachment-backed work products the way the shipped card does (gallery, text tab, new tab) with open and download beside the name; other work products are a link row with their status. - Wire the task thread: `TaskChatBubble` hangs a message's files under its footer as V1 rows inline (pictures and video to the gallery, text to a text tab, files to a new tab) in place of the media grid and file chips; `TaskChatProtocolCard`'s resource rows render deliverables, documents and attachments as one V1 row each, opening where the card did. - `ArtifactRow` keeps trailing controls beside its own control (no nested interactive elements) and can be a link, an in-app anchor, a new-tab anchor or a button; adapters read work products, documents, attachments and message references. - Add a catalogued Storybook story, `Components/Artifact V1 primitives`, under `documents-artifacts`, on fixtures. - Add tests: the primitives (kinds, dates, print fitting, tiles, rows), the hook, the Artifacts page, the panel's Artifacts tab, the task-chat bubble and the protocol card, each with the setting on and off; the settings page and server tests list the new key. - Add `doc/plans/2026-10-22-artifacts-mvp-v1.md`. ## Verification - Run `cd ui && pnpm typecheck`: the Storybook catalog accepts the new story, and `tsc -b` is clean apart from the five pre-existing `PluginOrganizationSwitcher` errors. - Run `pnpm check:token-gates` and `node scripts/check-maps.mjs`: clean. - Run `cd ui && npx vitest run src/components/artifacts src/components/issue-properties src/components/task-chat/TaskChatBubble.test.tsx src/components/task-chat/TaskChatBubble.artifactsV1.test.tsx src/components/task-chat/TaskChatProtocolCard.test.tsx src/components/task-chat/TaskChatProtocolCard.artifactsV1.test.tsx src/pages/Artifacts.test.tsx src/hooks/useArtifactsV1Enabled.test.ts src/pages/InstanceExperimentalSettings.test.tsx`: the suites pass (151 tests in the last run). - Run `cd packages/shared && npx vitest run src/feature-catalog.test.ts` and `cd server && npx vitest run src/__tests__/instance-settings-service.test.ts`: pass. - Manual: open Settings > Experimental, turn on Artifacts V1. On the Artifacts page the view row reads Tasks, All files, Images, Videos, Documents, Text, Files; tasks with several artifacts show as stacks, hovering fans the pages and lifts the artwork (not under reduced motion); Images shows single tiles with bordered prints. Open a task's Artifacts tab: one list of rows, newest first; a Markdown work product still expands into its review surface, a document still opens in its tab. In the thread, a message's files sit under its footer as rows. Turn the setting off: everything returns to the current presentation. - Storybook: `Tasks and conversations / Documents, attachments, and work products / Components / Artifact V1 primitives`. - Not run: the full `pnpm test:run` and `pnpm build` (CI runs them), and a browser pass against a dev instance with real artifacts on the three surfaces; that pass is the remaining step before this leaves draft. ## Risks - Off by default; with the setting off, the only changes on a product path are an optional `filters` prop on the toolbar, an optional `face` prop on two panel rows, a read of the setting in the bubble and the protocol card, and the new token declarations. - With the setting on, the Artifacts page's view tabs set two URL parameters (`groupBy`, `kind`) at once; the group-by menu and search are unchanged. - A picture with unknown dimensions renders at 4:3 until it loads, then the print refits once; the frame's height is fixed so nothing below it moves. - About thirty new `--artifact-*` tokens, namespaced and documented in place. ## Model Used - Claude Fable 5.1 by Anthropic, model ID `claude-fable-5-1`, used through Claude Code in the desktop app. Extended reasoning was on. Capabilities used: tool use (shell, file edits, a headless Playwright browser for the lab's measurements, the Figma MCP server for the mocks during the lab work) and image input. The context window is the Claude Code session's; its exact size is not surfaced to the session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (#17213 is the lab this builds on) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (the suites listed above; CI runs the full set) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the plan document; no command or product behaviour changes outside the setting) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
4dfcb4c9b8 |
refactor(tools): isolate connection access and removal (#17167)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connection grants and installations control which identities and agents can reach tools. > - Removal must close those access paths and stop runtimes before cleaning up owned secrets. > - These responsibilities still shared the setup coordinator after earlier extractions. > - This pull request gives grants, installations, credential validation and removal explicit owners. > - The benefit is a smaller coordinator and focused access-lifecycle modules with existing security ordering intact. ## Linked Issues or Issue Description Depends on #17165. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → #17162 → #17165 → this PR. **What existing behavior does this improve?** Maintenance of connection audiences, delegation, installation and removal. **Current behavior** The 6,419-line service still owns grant mutation, installation reach and resumable credential cleanup alongside app setup. **Proposed behavior** A grant owner manages audiences and delegation. An installation owner manages reach and its profile bindings. A removal owner keeps access revocation, runtime shutdown, secret classification and cleanup together. The coordinator delegates through typed APIs. **Reason and benefit** The entry point falls to 4,821 lines / 198,065 bytes, meeting the requested under-5,000-line target. Across the full stack it falls from 21,525 lines / 829,519 bytes. App setup remains together in the coordinator. ## What Changed - Extract credential validation (75 lines), grants (603), installations (350) and removal (658). - Preserve company boundaries, audience advisory locks, membership locks, assignment locks and audit order. - Keep the existing runtime supervisor and secret client. Retain live HTTP options for provider shutdown. - Keep Slack credential-mutation leasing and resumable owned-secret cleanup with removal. ## Verification - Removal, grant-resolution and the stronger real HTTP OAuth fixture pass before extraction (13 cases); the existing service and publication baselines also pass. - The original combined targeted run passed 525 cases across seven suites. After replaying onto master, 601 cases pass across nine suites, including Braintrust lifecycle and connector-policy coverage, with one database worker. - Full repository typecheck and build passed before merge integration. After integration, server typecheck, module-boundary and whitespace gates pass; fresh CI covers the full repository gates. - All 94 pre-existing function/method bodies match by normalized syntax comparison. A full-stack TypeScript symbol audit preserves all 18 exports and 89 service methods. - The earlier local full-suite run was interrupted after 79 minutes and remains unverified. All current-head CI gates pass on `d5307fcea2365e51a75d49db82deeed236d19951`, including the full sharded test matrix, browser suites, typecheck, build and canary dry run. Greptile is 5/5 with no open review threads after merge integration. - GitHub canceled duplicate CI run [39716841632](https://github.com/paperclipai/paperclip/actions/runs/39716841632) during the atomic stack update. Replacement run [39716843243](https://github.com/paperclipai/paperclip/actions/runs/39716843243) passed on the same head; no failing test was retried. ## Risks - Audience replacement must not erase an inactive member marker and widen authority. Its advisory and membership locks remain in the grant owner. - Uninstall must remove only its own profile bindings. Existing operator-authored bindings keep their current behavior. - Removal must remain fail-closed and retryable after provider failures. Its order, shared-secret retention, supervisor and cleanup references are unchanged and covered by removal tests. - No schema, public API, policy or scheduling change is intended. The first eight PRs have landed; this PR follows the updated catalog extraction. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
f7210cd31f |
refactor(tools): isolate catalog ownership (#17165)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive reviewed actions discovered from their connections. > - Transport, discovery, risk classification, publication and account inventory lived in the same service. > - Their ownership differs: transport guards requests, discovery reads providers, and publication changes the catalog under locks. > - This pull request gives those phases separate modules with explicit capabilities. > - The benefit is a visible discovery-to-publication path that preserves permission and quarantine behavior. ## Linked Issues or Issue Description Builds on merged #17162 and preserves the connector policy from #17141. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → #17162 → this PR. **What existing behavior does this improve?** Maintenance of tool discovery, catalog publication, health and upstream account inventory. **Current behavior** The 8,988-line service handles HTTP guards, MCP sessions, provider probes, risk classification, quarantine, cached reads and account sync. **Proposed behavior** Discovery returns descriptors through the existing credential and transport owners. Publication owns catalog writes and default access. A reader owns refresh deduplication. Health and aggregator inventory have separate owners. **Reason and benefit** The entry point falls to 6,419 lines, a reduction of 2,569. Provider discovery no longer shares a closure with connection setup and teardown. ## What Changed - Extract guarded remote HTTP, descriptor policy, discovery, publication, default access, cached reads, health, attention and aggregator account sync. - Keep the public descriptor/risk exports and existing service methods. - Preserve database locks, transaction-scoped default-access writes, quarantine decisions, current grant checks and failure classification. - Preserve per-service catalog flights and aggregator jobs, lease ownership and live remote-request options. - New modules range from 39 to 492 lines. ## Verification - Before and after the move, 470 public service/HTTP integration cases and 44 publication, HTTP rebinding and aggregator sync cases pass. - Server typecheck, module-boundary gate and whitespace checks pass. - All 157 pre-existing function/method bodies match by normalized syntax comparison. - Before merge integration, the completed stack passed 525 targeted cases, repository-wide typecheck and build. Its stronger rotating-token fixture is included. - After replaying onto master, the integrated stack passes 601 cases across nine suites, including Braintrust lifecycle and connector-policy coverage. Server typecheck and module-boundary checks pass. All 157 moved bodies still match the current master versions. - The earlier local full-suite run was interrupted after 79 minutes and remains unverified. All current-head CI gates pass on `a21595f20809a514b7619d8a8748c56f8e546e2f`, including the full sharded test matrix, browser suites, typecheck, build and canary dry run. Greptile is 5/5 with no open review threads after merge integration. ## Risks - Discovery must keep its current grant and session behavior; publication must retain its locks and quarantine rules. Existing authority, schema-drift and publication-race tests exercise these paths. - Aggregator account observations must not become authorization. Lease, freshness, failure and credential-fingerprint behavior stay unchanged. - Live transport options must not be snapshotted. The HTTP owner retains the same options object. - No schema, public API, access-policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ba9e6e4bc5 |
refactor(tools): isolate authorization lifecycle (#17162)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Browser authorization binds a consenting identity to a company connection. > - Direct OAuth, managed-cloud claims and Vercel authorization have different state and persistence rules. > - Keeping all of them in the general tool service obscures those rules and their cleanup owners. > - This pull request separates state, initiation, provider completion and explicit identity promotion. > - The benefit is reviewable authorization flows with their existing authority checks and ordering preserved. ## Linked Issues or Issue Description Depends on #17161. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → #17161 → this PR. **What existing behavior does this improve?** Maintenance of browser authorization and credential-sharing lifecycle. **Current behavior** The 12,019-line service combines provider callbacks, actor-bound states, authorization URLs and personal-to-company credential promotion. **Proposed behavior** Separate owners handle authorization state, start, direct/cloud/Vercel callbacks, catalog completion and explicit access finalization. Shared setup policy and identity checks have typed APIs used by the coordinator and authorization owners. **Reason and benefit** The entry point falls to 8,997 lines, a reduction of 3,022. Provider-specific state consumption and durable-write rules are visible in their own modules. ## What Changed - Extract authorization flows and their constraints; the largest new module is 700 lines. - Preserve direct/Vercel terminal state consumption and managed-cloud retryability until claim persistence commits. - Keep membership and grant locks, secret transaction clients, interaction resolution and catalog ordering unchanged. - Keep partial secret acquisition and cleanup together in access finalization. - Give shared organization-grant persistence, personal identity checks, setup policy and response projections named owners. ## Verification - All 470 public service and real local HTTP/PostgreSQL OAuth cases pass before and after extraction. - Coverage includes same-actor state binding, replay/denial, scope selection, membership revocation, provider failures, refresh concurrency and persisted usable credentials. - Server typecheck, module-boundary gate and whitespace checks pass. - All 182 pre-existing function/method bodies match by normalized syntax comparison. - The completed stack passes 525 targeted cases, full repository typecheck and build. The earlier interrupted local full suite remains unverified; all current-head CI gates pass; Greptile is 5/5 with no open review threads. - Original CI failure retained: [server shard 1/12](https://github.com/paperclipai/paperclip/actions/runs/39710536789/job/119600621627), `workspace-runtime.test.ts` case “replaces a reused Paperclip dev runtime whose 2xx health payload is unhealthy”. The fixture failed its initial health readiness with `ECONNREFUSED` after 97 probes / 10 seconds. The fixture and runtime implementation are unchanged and do not import this service. The isolated case passes without changing its timeout or assertions. One retry of only that shard passed at the same commit. All CI gates are now green; the original failure remains recorded here. ## Risks - Provider flows intentionally consume authorization state at different points. This extraction preserves each flow rather than sharing its completion algorithm. - Promotion must remove partially created secrets on failure and retire source secrets only after persistence. Both phases remain in the same owner with unchanged ordering. - Local HTTP fixtures prove assembled application behavior, not live third-party qualification. - No schema, public API or authorization policy change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
4fdef9457f |
refactor(tools): isolate credential ownership (#17161)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections need durable credentials that survive authorization and token rotation. > - Storage, registration, refresh coordination and credential selection lived in the general tool service. > - Each has a different ownership boundary: secrets, process-wide registration, database leases and per-service refresh work. > - This pull request gives those responsibilities named modules and passes their narrow capabilities to consumers. > - The benefit is visible credential ownership while retaining current concurrency and failure behavior. ## Linked Issues or Issue Description Depends on #17159. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → #17159 → this PR. **What existing behavior does this improve?** Maintenance of OAuth credential persistence, refresh and provider credential access. **Current behavior** The 14,877-line service owns registration flights, refresh leases, secret writes and provider selection alongside app setup. **Proposed behavior** Dedicated owners handle credential storage, client registration, refresh leases, connection/grant refresh, GitHub continuity and Vercel credential access. Credential selection composes those owners. Authorization callbacks remain in the coordinator for the next extraction. **Reason and benefit** The entry point falls to 12,019 lines, a reduction of 2,858. Lease acquisition and cleanup can be reviewed together, independently of callback completion. ## What Changed - Extract credential owners and shared single-flight/connection-link primitives. The largest new module is 650 lines. - Preserve process-wide client-registration flights, per-service refresh maps and GitHub sweep state. - Preserve live cloud-connector lookup, options and environment reads; use the existing secret client and transaction-scoped clients. - Extend the real local HTTP/PostgreSQL OAuth fixture with concurrent refresh across two service instances and authenticated use of rotated credentials. ## Verification - The two extended HTTP integration cases pass before extraction for public and client-secret-basic clients. - After extraction, all 470 public service and HTTP integration cases pass, including refresh failure, abandoned leases, secret redaction and membership revocation cases. The fixture was then strengthened after review: a second expiry must use the persisted rotated refresh token, and the provider rejects the old token. Both cases pass before and after extraction; server typecheck also passes. - Server typecheck, module-boundary gate and diff whitespace checks pass. - All 233 pre-existing function/method bodies match by normalized syntax comparison. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review findings. - Prior-head CI failure retained: run 39708550066, general server shard 2/12, `native-session-executor.test.ts` case “preserves a real listening process and its marker when a health RPC times out” raised `runner_remote_process_identity_unavailable` / `process_identity_changed`. Both the fixture and runner implementation are unchanged. The isolated case passes without changing assertions or timeouts. The fresh full CI run passes for the test-only review fix; the prior failure remains recorded here. ## Risks - Changing state lifetime could permit duplicate refreshes or registrations. Existing concurrency cases and the real HTTP fixture exercise these paths. - A transaction-scoped secret write must stay inside its authority lock. Function bodies, await ordering and client arguments are preserved. - The fixture proves assembled local HTTP/database behavior, not live third-party qualification. - No schema, public API, policy or scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ffcd113761 |
refactor(tools): isolate runtime and activity ownership (#17159)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators inspect tool execution, review pending actions and control local runtimes. > - These paths shared a large service with token exchange and app authorization. > - Runtime state and execution history need explicit owners and stable public responses. > - This pull request separates runtime controls, health summaries, activity reads and the review queue. > - The benefit is focused modules that preserve the current supervisor and approval behavior. ## Linked Issues or Issue Description Depends on #17157. Earlier stack: #17144 → #17146 → #17147 → #17155 → #17157 → this PR. **What existing behavior does this improve?** Maintenance of tool runtime administration and execution activity views. **Current behavior** The 16,100-line service contains runtime slot controls, health calculations, usage history, invocation decisions and action review queries. **Proposed behavior** Runtime modules own slot persistence, operator controls and health summaries. Activity modules own history, response projections and the review queue. The public service delegates to these owners. **Reason and benefit** The entry point falls to 14,877 lines, a reduction of 1,223 lines. The coordinator no longer needs execution-history joins or runtime alert calculations. ## What Changed - Extract five modules totaling 1,282 lines. The largest is 426 lines. - Pass the existing supervisor to slot controls. Pass the same live deployment options, clock and host getter to runtime health. - Keep activity reads scoped as before. Keep unsigned-request grace and invalid-signature cancellation in the review-queue owner. - Preserve public methods, response projections, query order and audit writes. ## Verification - The public service and unsigned review-queue tests pass before and after the move. The combined post-move run passes 469 cases with one database worker. - Existing cases exercise actual local stdio stop/restart, route permissions, stale-slot and audit-failure health, activity attribution, run decisions and unsigned pending requests. - Server TypeScript check, module-boundary gate and `git diff --check` pass. - Normalized syntax comparison finds all 256 pre-existing function/method bodies unchanged. - Full typecheck/build will run at the stack handoff. The earlier interrupted local full suite remains unverified; all current-head CI gates pass, with Greptile 5/5 and no open review threads. ## Risks - Creating another supervisor would split process ownership. The new module receives the same instance; it does not construct one. - Copying deployment settings could freeze live values. The health reader holds the existing options object and getter. - Review-queue reads can cancel invalid requests. Those writes and the unsigned-request grace period are preserved and covered by the existing regression test. - No schema, public API, authorization policy or background scheduling change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a42f213244 |
refactor(tools): isolate connection token broker (#17157)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents can request short-lived tokens for connections that allow token exchange. > - Admission, scope checks, exchange protocols and issuance records lived inside the general tool service. > - The security flow needs one visible owner while provider payloads need a separate boundary. > - This pull request extracts the broker into admission, configuration, endpoint, exchange and issuance modules. > - The benefit is a focused issuance flow with its locks, authority checks and audit order intact. ## Linked Issues or Issue Description Depends on #17155. Earlier stack: #17144 → #17146 → #17147 → #17155 → this PR. **What existing behavior does this improve?** Maintenance and verification of connection token issuance for agent runs. **Current behavior** A 17,752-line service contains broker configuration, endpoint policy, credential exchange, admission and issuance. **Proposed behavior** A dedicated broker coordinates the same admission and transaction flow. The exchange owner returns normalized token material. It receives the existing secret client, including the transaction-scoped client at issuance time. **Reason and benefit** The entry point falls to 16,100 lines, a reduction of 1,652 lines. Reviewers can see the responsible-user lock and hash-only issuance write without navigating app setup or OAuth callbacks. ## What Changed - Extract six broker modules. The largest is 755 lines; together they contain 1,721 lines. - Give tool-access audit persistence a named owner shared by the broker and coordinator. - Keep current-state checks, policy decisions, rate limits, secret resolution and transaction order unchanged. Private-host configuration is still read live. - Add two public-route integration cases with a real local HTTP server and disposable PostgreSQL. Cover Pages and RFC 8693 success and upstream failure. ## Verification - Baseline: all 466 existing public service tests passed before this move. The two new real HTTP cases passed on the original implementation before extraction. - After extraction: 495 tests pass across the public service and HTTP rebinding suites, with one database worker. This includes the two new cases, membership revocation ordering, scope denials, disabled broker behavior and hash-only persistence. - Server TypeScript checking, the module-boundary gate and `git diff --check` pass. - Normalized syntax comparison finds all 294 pre-existing function/method bodies unchanged. - Full typecheck and build passed for the preceding batch. Its interrupted local full-suite result remains unverified. All current-head CI gates pass; Greptile is 5/5 with no open review threads. ## Risks - A new closure could use the wrong secret client or move exchange outside the membership lock. The existing transaction still passes `secretService(tx)` to the exchange owner after taking that lock. - Endpoint validation must preserve live configuration and fail before credential disclosure. Existing rebinding and private-host tests cover this path. - The local fixture proves real HTTP and persisted application behavior. It does not qualify a live third-party broker. - No schema, API or access policy change is intended. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
233753e577 |
refactor(tools): isolate profile ownership (#17155)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tool profiles select the actions each agent can use. > - Profile reads, review, mutation and assignment resolution shared a large OAuth and connection service. > - Those paths have their own company checks, lock order and database projections. > - This pull request gives profiles explicit owners while preserving the existing transactions and public API. > - The benefit is a smaller coordinator and a reviewable access boundary. ## Linked Issues or Issue Description Depends on #17147. Stack: #17144 → #17146 → #17147 → this PR. These PRs remain open. **What existing behavior does this improve?** Maintenance of tool profile management and effective agent assignments. **Current behavior** The service entry point contains 19,293 lines. Profile behavior is spread across its closure and returned object. **Proposed behavior** Profile management accepts only a database and clock. Assignment resolution accepts three named capabilities. Projection and shared lock functions have explicit homes. Public methods and responses remain unchanged. **Reason and benefit** Reviewers can follow profile transactions, lightweight reads and precedence without navigating OAuth setup. The entry point falls to 17,752 lines, a reduction of 1,541 lines in this PR. ## What Changed - Extract profile management, projections and effective assignment resolution into three modules. Their sizes are 886, 268 and 216 lines. - Move shared actor/transaction types, identifiers and assignment locks into named modules. Preserve the public lock export. - Keep all database reads, transaction boundaries, lock order and method signatures. Add no cache or background work. - Record the next ownership cuts toward an entry point below 5,000 lines. ## Verification - Before and after: 466 cases in `tool-access-service.test.ts` and 33 cases across profile entry validation, effective assignments and catalog publication. All 499 cases pass with one database worker. - Server TypeScript check and `node scripts/check-module-boundaries.mjs` pass. `git diff --check` passes. - Normalized syntax comparison finds all 329 pre-existing function/method bodies unchanged. Wiring passes the unchanged public service tests. - The previous stack passed full typecheck and build. Its full local test run was interrupted and remains unverified; its complete CI matrix passed. This PR will record its own current-head CI result below. - Current-head CI is green. Current review: Greptile is 5/5 at `0f5c85905e`, with no inline review threads. - CI failure preserved: [general server shard 1](https://github.com/paperclipai/paperclip/actions/runs/39704932041/job/119582710174) failed the unchanged workspace-runtime test that replaces an unhealthy local process. The replacement refused connections for its original 10-second readiness budget. The fixture and runtime service are unchanged and do not import tool-access. One isolated run passed without changes or timeout increases. The failure log is preserved; the cause is not reproduced. Only that failed job was retried; it passed at the same head without code changes. The original failure remains recorded above. ## Risks - Moving profile mutations could change transaction clients or lock order. Both are unchanged and the existing database tests cover publication and assignment behavior. - Effective assignment resolution still uses the same installed-connection loader and full catalog projection. The lightweight path still omits large schema and metadata fields. - No schema, public API or access policy change is intended. Existing tests provide characterization coverage; no new test mirrors the extraction. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
4413bb262f |
refactor(tools): isolate OAuth protocol boundaries (#17147)
## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections use OAuth to obtain credentials from remote providers. > - The tool-access service mixed provider protocol parsing with database and secret writes. > - This made protocol changes hard to review without reading the full coordinator. > - This PR gives discovery, endpoint policy, and client exchange explicit owners. > - The coordinator keeps authorization, leases, persistence, and audit decisions. > - A real local provider fixture checks the boundary through public routes. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership of OAuth discovery, registration, token exchange, and client metadata. **Current behavior** The tool-access service contains over 20,000 lines. Provider responses, endpoint policy, and stateful operations share the same closure. **Proposed behavior** Small protocol modules accept settings and named request capabilities. They return normalized results. The service owns durable state and existing public exports. This is part 3 of the stack: #17144, then #17146, then this PR. It targets #17146's branch. Review only this PR's diff. ## What Changed - Extract endpoint policy, metadata parsing/discovery, registration/token exchange, provider error normalization, and the public client metadata document. - Validate remote registration responses in the protocol module. Return client material to the coordinator for secret and connection writes. - Remove the unused raw provider response from the internal token result. Keep every consumed field. - Preserve process-wide registration single-flight, service-local maps, refresh leases, membership checks, callback checks, and transaction ownership. - Reuse connection-method URL normalization. Keep the endpoint setting and clock callbacks live. - Add a real local HTTP OAuth/MCP route fixture for public and Basic-authenticated clients. - Reduce `tool-access.ts` from 20,917 lines / 808,777 bytes to 19,293 lines / 749,158 bytes. The seven OAuth modules total 1,697 lines; the largest has 582 lines. The stack removes 2,232 lines from the original service. - Record the next profile-resolution and token-broker boundaries in the dated plan. ## Verification - Baseline: 466 service cases and 80 protocol/provider-policy cases pass. The new two-case HTTP fixture passes before moving production code. - After: `pnpm exec vitest run server/src/__tests__/tool-access-oauth-protocol.integration.test.ts server/src/__tests__/tool-access-service.test.ts server/src/__tests__/remote-mcp-connectors.test.ts server/src/__tests__/remote-http-rebinding.test.ts server/src/__tests__/oauth-client-metadata.test.ts` — 508 tests pass with one worker and disposable PostgreSQL. - The fixture drives connect, metadata discovery, DCR, PKCE code exchange, callback, catalog review, and finish. It checks persisted active state, secret references, issuer/auth-method binding, and wire-level authentication. - `pnpm -r --workspace-concurrency=2 typecheck` passes. The sandbox-only attempt could not open a local IPC socket; the same command passed with local process permissions. - `node scripts/check-module-boundaries.mjs` and `git diff --check` pass. - AST comparison confirms 37 moved bodies are unchanged. Registration protocol and persistence statements match their original halves. Token exchange only drops its unused raw result. Preflight uses explicit defaults and a clock callback. - Initial fixture authoring used a nonexistent catalog enabled field; the corrected assertion checks persisted status and reviewer identity. An initial typecheck found optional defaults at the new boundary; the caller now passes its already-validated defaults. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. Remote CI is green at `3dc46c132408556ecb31038a9b9621ac20aacefc`. Greptile is 5/5 with no actionable findings or open inline threads. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - OAuth endpoint checks and issuer/resource binding are security boundaries. Existing denial, redirect, scope, DCR, secret-redaction, and refresh tests cover the preserved paths. - A moved closure could capture stale policy or split registration/refresh ownership. The new modules hold no mutable state. Policy is read through the existing callback, and state stays in the service. - The local fixture proves real HTTP and database behavior. It does not claim a live third-party provider sign-in. - No schema, API, authorization policy, or persisted-data migration changes are intended. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
8e95caf9ae |
refactor(tools): isolate connection method projection (#17146)
## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Connection methods define operator settings and managed tool arguments. > - Setup, catalog discovery, and gateway invocation must use the same projection rules. > - These rules were embedded in a large database service. > - This PR gives them one module with a narrow settings contract. > - Existing provider policy modules still decide their provider-specific restrictions. ## Linked Issues or Issue Description **What existing behavior does this improve?** Connection method configuration and tool input projection. **Current behavior** The tool-access database service owns settings validation and pure input/schema transforms. Callers import these through the service. **Proposed behavior** A connection-method module owns these transforms and accepts only connection settings. Existing imports remain compatible. This PR depends on #17144 and targets its branch. Review only this PR's diff. It is part 2 of the three-PR stack. #17147 follows this PR. ## What Changed - Move method selection, settings validation, and header/schema/argument projection into `tool-access/connection-methods.ts`. - Replace database row parameter types with required config and transport settings. No new validation is added inside typed calls. - Keep Honcho, Google Chat, and Gmail policy in their existing modules. Preserve their call order. - Retain the service's public exports for routes, gateway consumers, and tests. - Reduce `tool-access.ts` from 21,237 lines / 820,116 bytes to 20,917 lines / 808,777 bytes. The new module is 359 lines / 12,665 bytes. ## Verification - Baseline: all 466 tool-access service tests and 80 protocol/provider-policy cases pass before the move. - After: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts server/src/__tests__/memory-connectors.test.ts server/src/services/google-chat-tool-policy.test.ts server/src/services/gmail-tool-governance.test.ts` — 506 tests pass with one worker and disposable PostgreSQL. - `pnpm --filter @paperclipai/server exec tsc --noEmit` passes after the existing workspace prerequisites were built. - AST body comparison confirms the 11 moved functions and private record parser are unchanged. `git diff --check` passes. - The first typecheck found optional config in the public DTO. The new input type now states the existing required-config invariant directly. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. Greptile is 5/5 at `2d0bb86b5830c0306580e3ddc8637c4994b2a75a` with no actionable findings. - CI passed all non-browser gates. [The first browser attempt](https://github.com/paperclipai/paperclip/actions/runs/39675461620/job/119484152569) failed in the unchanged Cases read-only test: the status filter remained All statuses while the announcement-dismiss locator handler interrupted the assertion. The same test passed on #17144. The failure log is preserved. One retry of only the failed jobs passed without code changes; all CI gates are now green on the same head. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - A projection-order change could alter managed arguments or advertised schemas. Function bodies and call order are unchanged, and tests exercise setup, discovery, and gateway consumers. - Legacy Gmail method aliases, GitHub toolset headers, missing unrelated settings, and disabled method filtering retain their previous behavior. - The module has no database dependency, new network work, or mutable state. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9db3a5726e |
refactor(tools): isolate GitHub access discovery (#17144)
## Thinking Path > - Paperclip manages AI agents and their access to external tools. > - Tool connections discover repositories through GitHub account grants. > - The tool-access service mixes GitHub response parsing with grant authorization and persistence. > - These responsibilities have different inputs and failure rules. > - This PR puts GitHub discovery in one provider module. > - The service keeps current authority checks and stored grant updates. ## Linked Issues or Issue Description **What existing behavior does this improve?** The ownership and review of GitHub access discovery in the tool-access service. **Current behavior** The 21,525-line service contains GitHub pagination, account metadata validation, and grant lifecycle code. **Proposed behavior** A GitHub module returns normalized access metadata. Existing service exports remain compatible. This is the first PR in a three-part refactor stack. No behavior change is intended. A search found no existing tool-access refactor PR or issue. Stack order: this PR, #17146, then #17147. Each PR targets its predecessor. ## What Changed - Move six functions into `tool-access/github-access-discovery.ts`. Their bodies are unchanged. - Keep grant selection, credential resolution, refresh retry, membership locks, revision checks, and webhook updates in the service. - Add public-entry characterization tests for HTTP error classification and invalid pagination. - Record five candidate boundaries and the initial three-PR scope in the dated plan. - Reduce `tool-access.ts` from 21,525 lines / 829,519 bytes to 21,237 lines / 820,116 bytes. The new module is 294 lines / 9,722 bytes. ## Verification - Before and after: `pnpm exec vitest run server/src/__tests__/github-grant-metadata.test.ts server/src/__tests__/project-repositories.test.ts` — 13 tests pass. - Before and after: `pnpm exec vitest run server/src/__tests__/tool-access-service.test.ts -t 'GitHub|github'` — 11 selected tests pass with disposable PostgreSQL; 455 cases are outside this selection. - `pnpm --filter @paperclipai/server typecheck` passes. - Exact text comparison confirms all six moved bodies match the base. `git diff --check` passes. - The initial baseline attempt failed because the plugin SDK had not been built. The normal workspace prerequisite build resolved it before behavioral validation. - Final-stack local verification: `pnpm -r --workspace-concurrency=2 typecheck` and `pnpm build` pass. CI is green at `07e78dd1b1c26e0bd4234006766444c71e234451`. Greptile is 5/5 with no actionable findings. - Full local-suite limitation: `pnpm test:run` was interrupted after 79 minutes during the serial general-server phase, after CI had completed the full test matrix. Before interruption, it flagged the unchanged `company-search-service.test.ts` suite (26 database cases skipped). One isolated rerun passed all 28 cases without code changes. The original log and a pre-interruption copy are preserved. The original failure cause was not reproduced. The full local suite and its later groups are not claimed as passed; the complete CI matrix is green on this stack. ## Risks - Import wiring could break existing consumers. Existing imports remain supported and all four production call sites are unchanged. - Provider requests retain their ordering, timeout, fixed GitHub origin, and response redaction. The new module has no database dependency or shared mutable state. - Tests simulate GitHub responses. They do not qualify live GitHub access. ## Model Used OpenAI Codex, GPT-6 family, with code execution and repository tools. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
92cfd701c3 |
feat: add governed Braintrust app connections (#17141)
Add hosted US and EU Braintrust OAuth and API-key connections through the existing governed MCP system. Keep reviewed tool risks and catalog review policy in connector modules and a narrow registry. Preserve credential ownership, operator decisions, regional setup and changed-tool quarantine. Include official artwork, setup guidance, focused regressions and sanitized qualification evidence. US OAuth has bounded live read/write, gateway, reconnect and cleanup proof. Remaining account-method and token-lifecycle qualification gaps are documented. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
efc2ddc7bf |
fix: stop idle login cleanup polling (#17106)
**Dependency landed:** PR #17129 is merged. This branch is rebased onto master `0f15f82b19` and preserves its provider-owned completion contract. The shared save path calls `notifyLoginSessionChanged(tx)`; the cleanup module invalidates both existing sandbox cleanup workers without selecting a provider. ## Thinking Path > - Paperclip manages work for AI agents. > - Subscription login can allocate a temporary sandbox. > - Existing cleanup workers scan the database every heartbeat tick, even with no login work. > - Empty scans keep idle databases active. > - This PR uses committed session changes and saved expiry times to schedule cleanup. > - Existing rows still drive recovery after failure and restart. ## Linked Issues or Issue Description **What happened?** On master `898144f4b3`, the enabled heartbeat scheduler runs both login reapers every 30 seconds by default. An instance with no pending logins still queries sessions and leases. **What did you expect to happen?** After startup reconciliation, idle login cleanup should make no recurring SQL or provider calls. New sessions, expiry, failed cleanup and restart must still recover. **Steps to reproduce** Start a claimed instance with heartbeat scheduling enabled and no pending login sessions. Observe repeated adapter-login and setup-token cleanup queries at the heartbeat interval. Related sleep migration: #17094. Searched open issues and PRs. Related setup-token cleanup work: #15774, #14236 and #16648. This PR does not change their release-result or null-lease behavior. ## What Changed - Register separate adapter-login and setup-token workers with the existing delivery coordinator and scheduler. - Notify before writes in the owning stores, the lease-tag write, and AI connection save. Shared save uses a provider-neutral notification; each cleanup worker checks its own durable predicates. The shared transaction protocol retains intent until the outer transaction settles. - Reconstruct session expiry, promotion expiry and concrete cleanup retries from existing rows. Preserve configured heartbeat spacing to bound scan bursts. - Recover tagged orphan leases even with terminal or missing sessions. Protect live acquisitions that have not attached their lease yet. - Exclude local-subscription rows from sandbox reapers. Their existing local-login worker owns cleanup. - Preserve deferred work across drain and shutdown. Add no tables, Cloud alarms or host-sleep exemptions. - Update the inventory and document producer coverage, writer scope and recovery boundaries. ## Verification - Updated head: `1d57e13606`. Seven focused suites passed (282 tests): AI connections, login cleanup, device-login service/reaper, login-session runtime, and setup-token session/reaper. `pnpm -r typecheck` and `pnpm build` passed. The duplicate local `pnpm test:run` was interrupted after CI passed all server shards; it is not counted as a complete local pass. Full-suite evidence comes from the exact-head CI run. Greptile reviewed this exact head at 5/5 with no findings or unresolved threads. [Full CI run 39669285577](https://github.com/paperclipai/paperclip/actions/runs/39669285577) passed on attempt 2. Both aggregate gates passed; no checks remain failed or pending. - Extended the real PostgreSQL AI connection tests for both Anthropic and OpenAI. The tests check that neither cleanup observer wakes inside the completion callback, both wake after commit, and an idempotent save does not emit another wake. - Existing cleanup tests cover idle/history states, producer wake, expiry, promotion, cancellation, retry/restart, orphan recovery, nested commit/rollback, drain, concurrency, and shutdown. - Current-head CI attempt 1: the unchanged `sandbox-callback-bridge.test.ts` HTTP/2 preface test failed because its child process could not bind `127.0.0.1:34033` (`EADDRINUSE`). Its single unchanged rerun passed. The exact test also passed locally without changing code or timeouts. This establishes recovery from the bind collision; it does not establish which process owned the port. No login code ran in that package test. - Previous-head failures remain recorded: the login-runtime transaction fixture was corrected; a workspace-runtime readiness port-bind failure passed in isolation and fresh CI, but its first failure cause remains unconfirmed. An earlier combined AI suite timed out in local-login isolation and passed alone; its timing cause remains unconfirmed. An earlier local full suite was interrupted and was not counted as passing. ## Risks - Process-local signals cover one active server with the instrumented shared database handle. Independent SQL writers or replicas need their own signal integration. - Provider resources that never became persisted tagged leases remain the responsibility of the unchanged general lease/spool recovery. Setup-token acquisition retains its required provider expiry. - Cleanup starts are bounded by the configured heartbeat interval. Writes during a sweep can cause a final follow-up scan before quiet. - Existing login and resource blockers still prevent unsafe host sleep. This is worker polling removal, not provider sleep qualification. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and test execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes` / `Refs` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal/instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
0a5f1546ee |
feat(cases): add reusable prompt workflows (#17114)
## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
6dda8cc2b7 |
fix: preserve Stop authority and qualify complete recovery journeys (#17105)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - Users need complete tasks, clear questions, retained context, and dependable recovery. > - Separate run tests do not prove that the complete user experience works. > - Successful retries can hide earlier failures and missing evidence. > - This PR extends the existing Product E2E harness with three explicit journeys. > - It adds a qualification checkpoint over the full retained result pool. > - Independent artifacts, state, rendered conversations, and delivery receipts establish each outcome. ## Linked Issues or Issue Description Refs #17088, #17089, and #17046. This PR repairs two Stop lifecycle defects and combines the qualification changes previously reviewed in #17095, #17090, and #17099. Daily operations remain with #17051. It adds no second scheduler or recovery engine. **What happened?** A completed run can still leave a user without useful output, an understandable question, or a working continuation. Existing isolated tests and best-attempt reports do not establish the combined behavior. **Expected behavior** A necessary question includes its explanation and resumes once after an answer. Stop prevents automatic continuation until the user resumes. Restart during finalization preserves the accepted provider result and delivers it once. Qualification requires independent evidence for six journeys on both agents through Slack and Paperclip. **Steps to reproduce** List the necessary-question, followup-stop-resume, and finalization-recovery suites using the documented selectors. Run one exact cell with the configured model and target. Supply a private qualification manifest to the existing report command to grade all retained attempts. Missing proof and incomplete coverage must fail. ## What Changed - Wake existing accounting in the same committed native Stop closure. A receipt that arrives before closure must still settle once; incomplete receipts remain held. - Require a later user message before native Stop can promote earlier saved input. Preserve those earlier messages for the authorized resumption. - Add explicit necessary-question cells. Verify the actual prompt and context, answer, one bound continuation, and the delivered document link and revision. - Add explicit follow-up, Stop, and later resumption cells. Verify a live owned provider turn, cancellation acknowledgement, at least five seconds without an automatic path, retained direction and workspace context, and an actual browser download. - Add explicit Daytona finalization-recovery cells. Stop the owned controller after public result acceptance and during workspace finalization. Require the original result, restored file hash, resumed operation, one provider turn, and one receipt-bound delivery. - Share the receipt-based delivery observer across both recovery journeys. Keep model prose separate from the authoritative delivered comment identity. - Add a 24-cell qualification checkpoint over every retained attempt. Require three fresh candidate passes per cell and later staging proof with exact source, runtime, model, settings, instructions, and independent evidence. - Reject absent or unsafe manifest evidence. Preserve failed attempts and incomplete observations in the normal reports. - Provide four explicit rotating daily selections. Refuse daily selection until all required live drivers exist. Keep scheduling and spending with the existing operations owner. - Document the exact selectors, evidence contract, limitations, and remaining staging work. ## Verification - Local Product E2E support: 3,554 passed with two existing skips; source prerequisites: 128 passed. E2E typecheck passes. The actual file-chip and Download-control browser regression passes. - Two real-database regressions fail before the production repair. The stopped coordinator leaves accounting asleep after a complete receipt; the saved queue can restart from pre-Stop input. The repaired settlement and cancellation cohort passes 66 checks. The complete 256-case continuation suite passes with stronger two-phase Stop expectations, and 66 native executor accounting/Stop safety cases pass. Server typecheck passes. - All required CI, repository build/typecheck/tests, and fresh Greptile 5/5 pass at the final head |
||
|
|
507a0d7098 |
test: verify repository delivery after real runner conversion (#17089)
## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
|
eece07a747 |
fix: repair the complete projectless repository setup chain (#17088)
## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> |
||
|
|
898144f4b3 |
fix: stop idle execution-control recovery polling (#17094)
Replace idle execution-control polling with workers driven by committed writes and persisted deadlines. Preserve startup recovery, bounded retries, and shutdown safety. Stabilize the Slack setup browser regression by waiting for the restored setup screen before simulating its webhook. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
|
dff63551fe |
fix: resume unfinished work after repository preparation (#17047)
## Thinking Path
> - Paperclip manages agents that perform work for a company.
> - Task repositories are prepared at normal run admission.
> - A running native task can stage a repository while its work remains
unfinished.
> - The controller did not offer that saved setup as a continuation.
> - The agent could add a timer or wait for a user restart.
> - This change uses the existing same-agent continuation and status
ledger.
> - The accepted explanation and proven conversation remain available
after setup.
## Linked Issues or Issue Description
Refs #17045. That prerequisite has merged. This PR targets master.
**What happened?**
A projectless native task can request a repository for the next
admission. An unfinished task has no server-selected repository
continuation. An explicit same-agent continuation also loses chat
provenance.
**Expected behavior**
An unfinished standard task can yield and resume through the existing
admission queue. Its explanation and final result retain the authorized
conversation. A task that only stages a repository can complete without
another run.
**Steps to reproduce**
1. Start a native standard task in a task directory.
2. Request an authorized repository through prepare_repository.
3. Yield with unfinished work and no explicit continuation.
4. Observe that the former driver rejects the yield, although the
pending repository is saved.
## What Changed
- Offer a saved same-agent continuation for the current task's pending
repository.
- Accept that current server continuation in the shared provider driver.
- Resolve chat provenance through applied status decisions, delivered
effects, and admitted wakes.
- Preserve accepted setup explanations without repeating an owned
question card.
- Document optional reproduction details and configured repository
discovery.
- Extend database regressions and workspace access maps.
- Preserve Copilot profile v52 and bind the shared continuation source
in v53. Include the active identity in the QA image fingerprint.
## Verification
- Five focused server suites: 100 checks pass.
- Shared continuation driver: 12 checks pass.
- Server typecheck passes.
- Map contract and whitespace checks pass.
- Existing API and driver assertions fail on the former implementation.
- Real dispatcher and queue cohort: 43 checks pass. The original
reply-loss regression fails before the dispatch receipt fix.
- Continuation driver and installation identity cohort: 62 checks pass.
- Regenerated capability inventory and contract checks pass.
- Final response precedence cohort: 109 checks pass. The stale-comment
case fails on the former implementation.
- All required CI checks, build, and both source reviews pass on
|
||
|
|
c2daf906d9 |
fix: preserve task continuation and question delivery (#17045)
## Thinking Path
> - Paperclip manages agent work and delivers its results.
> - One task can need several runs and a saved workspace.
> - A run can succeed while the task waits for its next run.
> - Chat fallback treated that success as completion and dropped useful
status.
> - Monitor runs also lost their conversation origin, and text-question
cards lost their prompt.
> - This change preserves the task outcome and exact delivery authority
across those boundaries.
## Linked Issues or Issue Description
**What happened?**
A Slack task requested a repository, yielded to a scheduled
continuation, and showed only a generic completion message. The next run
could not publish through the original conversation. A later text
question showed its title and Respond button without the question
context. Workspace recovery could commit the saved monitor result
without restoring its explanation.
**Expected behavior**
The original conversation shows the accepted continuation explanation,
receives the resumed result, and displays the full question when input
is needed. A successful provider run does not claim that unfinished work
is complete. Recovery delivers saved output once without another
provider result.
**Steps to reproduce**
Start an authorized chat task, prepare a repository for a later
admission, save a monitor wait, and let that monitor resume the task.
Repeat with workspace finalization interrupted and with a free-text
question. Inspect the originating conversation and the persisted output.
Related merged work: #16806 and #16975. The open monitor and recovery
changes found during duplicate search address different status and
liveness policies.
## What Changed
- Keep committed monitor summaries visible through the shared board and
chat response resolver. Preserve interaction ownership and comment
reuse.
- Recover a saved monitor explanation after finalization, with no
repeated provider result.
- Prove monitor publication origin from the admitted wake and matching
scheduling receipt. Match company, task, agent, requester, and schedule;
reject ambiguous sources.
- Include monitor continuations in the existing publication scan and use
the canonical task binding. Keep current transport authorization.
- Suppress completion fallback for yielded results. Explain missing
final output without claiming completion.
- Include question summaries, prompts, and help text in modal-backed
cards. Keep answered text private.
- Update execution semantics, capability maps, and the bounded
reliability plan.
## Verification
- Original source fails four composed PostgreSQL regressions: monitor
origin, yielded completion, question body, and saved-summary recovery.
The shared response resolver also fails before the repair.
- Repaired focused cohort: 148 tests pass across publication, question
privacy, scheduler, and native monitor suites. These use real disposable
PostgreSQL; they do not qualify live Slack.
- Full workspace typecheck and production build pass locally. Map checks
and whitespace checks pass.
- All CI gates pass at
|