371 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	b40e6edaea	perf(inbox): combine startup reads and reduce refetch pressure (#18790) ... ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Mine shows the work and failures that need the viewing user's attention. > - A reload waits for account access, task rows, parent captions and failure state. > - Those reads form several browser request stages before useful rows appear. > - Live invalidations can also restart the same task-list request more than once. > - This change supplies startup data together and removes redundant refresh work. > - It preserves current authority, failure presentation, archive state and row stability. ## Linked Issues or Issue Description Refs #18645. Related: #18642. This change implements the read-receipt behavior independently on current master. It does not use that PR's branch or patch. **What happened?** Mine starts its primary request after the session, company and board-access reads. It then requests parent captions and rich failure context. Four observed loads spent another 0.45–0.70 seconds after the primary rows arrived. One slower load waited 1.71 seconds for parent context. The same run observed two HTTP 429 responses. These timings identify request waits; they do not attribute the delay to SQL or Node CPU. Read receipts also invalidate general task collections and dashboards. Overlapping parent and child invalidations can start two requests and abort the first. **Expected behavior** The initial page should use one account-safe startup projection and one complete Mine response. A read receipt should refresh the affected viewer's inbox state. Each invalidation batch should refresh a covered query once. Required captions and current failure state must be ready before first reveal. Existing rows should remain usable during a transient refresh error. **Steps to reproduce** 1. Open a populated Mine inbox with parent tasks outside the visible index and a current failed run. 2. Reload with browser HTTP cache retained and application query state empty. 3. Inspect the session, company, board access, task-index and context request sequence. 4. Open or mark a task read and inspect the unrelated collection refetches. **Paperclip version or commit** Baseline: `44a9a043b80c18314d8dd89556f444dae696d3d6`. **Deployment mode** Authenticated board UI. Local proof uses the real APIs and database with a synthetic session resolver. No hosted candidate deployment is claimed. ## What Changed - Add optional, same-user `boardBootstrap` data to the existing session response. Seed account-keyed company and board-access reads from it. Keep ordinary endpoint reads for fallback, retry and live reconciliation. A returning account cannot reuse revoked cached access. Cache collection and later real refetches remain effective. - Add the board-only Mine snapshot endpoint. Reuse the canonical issue selector, private-task filtering, run-read decisions, secret redaction and interruption/recovery presentation. Assemble the fixed 100-task page, parent captions and bounded 200-run failure window on the server. Start independent reads together. Optional source errors keep successful task rows usable. - Keep array-shaped source caches for archive and undo. Key Mine source, context and reveal state by account. Validate the response's user stamp before committing it. Remove array-only cross-tab Mine broadcasts, which cannot carry its paired failure context. The server retains authenticated in-flight coalescing. - Scope read-receipt invalidation to the affected viewer's inbox, badges and chat unread state. Keep company and task audit views fresh. Preserve genuine task mutations, run updates and subscription-gap reconciliation. - Remove covered scalar-prefix invalidations within a batch. Preserve predicates, exact matches and different query/refetch constraints. A later event can still refresh again. - Retire Mine in-flight reuse after local company activity or run lifecycle changes. A post-archive read starts fresh. Old work keeps its admission slot until it settles, even if every HTTP client aborts. The eight-compute limit stays in place. - Add API, component and browser regressions. Extend the production timing harness with matching API/UI artifact receipts, retained-cache reloads, request status counts, layout checks and optional controlled network latency. Update API and verification documentation. ## Verification - Full `pnpm -r typecheck` passes. - Full `pnpm build` passes. - Token gates, map checks and diff whitespace checks pass. - 72 Inbox component tests pass. Startup-specific checks pass, including account switching, revoked cached access, malformed/absent bootstrap data and cache collection. The canonical authenticated bootstrap route also passes against real PostgreSQL. - Six real PostgreSQL Mine route cases pass, including full equivalence with legacy authorized projections, privacy, fixed bounds, membership revocation, coalescing, archive freshness and aborted-client admission. Four focused legacy coordinator controls pass. Six assembly tests pass. - Refetch and invalidation regressions pass. Meaningful pre-fix tests observed two request starts and one abort for overlapping invalidations; the fix produces one start and no abort. The archive race test observed the post-archive GET joining the old body before the fix. - All 11 existing startup browser checks pass, including Tasks/Mine code loading, desktop/mobile bounds, non-streamlined Coordinator, task navigation, failed preloading and Back position. The nine focused startup, fallback, partial-source error, rate-limit recovery, archive/undo and geometry cases also pass. The 429 cases use the inherited retry window. Archive and keyboard undo are corroborated through the real API and reload. Authentication and failure fault injection in these cases are explicitly simulated. - Matched production before/after timings are complete. Full aggregate Vitest coverage is left to CI. At 15:15 UTC, the final-head runner gate and `review` workflow pass. The 45 build/test/policy jobs are queued; the Build job has no `ubuntu-latest` runner assigned and has executed no steps, so remote aggregate results remain unverified. Paperclip Review and Superagent Security Scan are in progress. Superseded runs for this PR were cancelled. No current-head check has reported a failure; the CI checklist remains open. The remaining step is runner allocation and completion of those external checks; persistent queueing needs the repository CI owner's attention. - Greptile completed on final head `ecde295565309275c4f2fae903caf6892f1eaafd` at 5/5. The map-scope finding is fixed and resolved; there are no open review threads. ### Matched browser timings Five alternating baseline/candidate pairs per network scenario, measured serially in headful Chrome 156.0.8078.80 on Apple silicon with Node 24.22.2 and a 1440×900 viewport. Both production UIs use their matching real API revision against one fixed disposable PostgreSQL fixture: 34 tasks, 31 visible task rows, one paused agent and one terminal failure. No active agent/provider work. HTTP cache is retained; each initial sample reloads the document and clears application query state. Warm navigation and Back retain query state. CPU and throughput are native. The 100ms scenario uses CDP minimum request-to-response-header latency without request interception. It is a controlled request-dependency experiment, not a staging reproduction. Values are median milliseconds with min–max in parentheses. Paired saving is baseline minus candidate for each alternating pair; its median differs from the difference of arm medians. Positive saving favors the candidate. | Network / journey | Baseline | Candidate | Median paired saving | | --- | ---: | ---: | ---: | | Native / initial rows | 187.6 (128.9–208.5) | 191.7 (130.3–210.9) | −23.3 | | Native / Recent → Mine | 100.17 (99.90–101.87) | 100.07 (99.88–101.37) | −0.16 | | Native / task → Back | 70.45 (56.63–72.02) | 66.64 (56.52–67.71) | +4.94 | | 100ms / initial rows | 930.3 (914.1–1015.7) | 799.0 (697.1–826.0) | +211.8 | | 100ms / Recent → Mine | 100.09 (97.73–100.16) | 100.18 (96.65–100.98) | −0.02 | | 100ms / task → Back | 74.97 (62.77–75.82) | 75.47 (61.51–76.19) | +0.35 | The controlled-latency initial pairs all improve, saving 108.3–217.0ms. Native initial pairs have mixed signs (−67.5 to +78.1ms), so they establish no reliable native page-speed improvement. Warm and Back timings are effectively unchanged. Median primary-request start is 110.4→135.2ms natively and 497.8→480.9ms with 100ms latency; these runs do not establish a large startup-only timing gain. API requests through the two-second post-reveal window fall from median 32 (32–37) to 27 (25–28) natively, and 37 (33–38) to 30 (25–32) with 100ms latency. All 20 samples preserve the same ordered 31 task links, first-row DOM identity and geometry. All recorded API responses are HTTP 200; zero unfinished requests, 429s or JavaScript errors. Lower 429 incidence on staging remains unproven. Measured source: baseline `44a9a043b80c18314d8dd89556f444dae696d3d6`, candidate `4c12b2ef0ecf0754676e99bd6b9399e84408646c`. Final head `ecde295565309275c4f2fae903caf6892f1eaafd` changes only two feature-map files and existing browser test expectations since that measured candidate; production executable inputs are identical to the measured candidate. Asset hashes, API health revision receipts and every raw sample were retained. These results do not claim hosted-page improvement or deployment of this PR. ## Risks - Startup enrichment adds work to the session read. Failure of this optional projection falls back to the ordinary company and access endpoints. It does not invalidate a valid session or replace per-request server authorization. - Account changes, stale cache entries and same-company navigation can expose old presentation data. Account keys, principal stamps, reveal scopes and canonical bootstrap replacement have dedicated negative cases. Permission failures hide cached task rows; transient network, 429 and 5xx failures preserve them. - Current failure state can differ from the historical failed run. The snapshot uses the existing canonical rich projection and verified missing-ID coverage. Partial context failures retain an explicit retry path. - Mutation retirement follows events delivered to the current server process. This adds no cross-replica invalidation protocol or distributed read-after-write guarantee. - New UI assets require the matching API. Older UI/session consumers retain their existing response and endpoint behavior. There is no database migration, provider call or deployment change. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code execution, browser automation and delegated source review. The exact runtime model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 09:28:57 -06:00
DottaandPaperclip	b22ef22d56	fix(runner): show provider failure details on tasks and runs (#18622) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native Runner starts providers and records their run outcomes. > - Provider failures can contain useful details below a generic protocol error. > - Those details were lost at the sidecar, command receipt, and run projection boundaries. > - A blocked task therefore showed a generic failure even when the provider gave a clear reason. > - This pull request carries a bounded, sanitized provider failure through those boundaries. > - Operators can read the actual cause on the task and run pages. Existing recovery policy still controls retries. ## Linked Issues or Issue Description Refs #17543 for the related closed JSON-RPC classification work. Refs #17272 for provider recovery in the legacy OpenCode adapter. This change adds display evidence for native Runner providers and keeps recovery policy separate. **What happened?** A native Claude session returned an HTTP 429 message below an ACP `Internal error`. The task and run showed only a generic startup failure. Operators had to inspect the provider response to learn that the account was rate limited. **Expected behavior** The task and run show the sanitized provider message, provider name, startup or turn phase, and available HTTP status. Unknown categories keep the message. Display evidence does not grant a retry or replace a recovery decision. **Steps to reproduce** 1. Start a native Claude run whose account is rate limited during model selection, or return a structured provider failure from a Codex or OpenCode turn. 2. Let the run fail and open its task page. 3. Before this change, the page shows a generic failure. With this change, the page shows the provider explanation. The run page shows the same explanation after reload. **Paperclip version or commit** Source checkout based on `f8a888a072`. ## What Changed - Add one JSON Schema for `ProviderFailure` and generate matching Runner and shared TypeScript readers. Keep the Runner package independent of the app workspace at runtime. - Normalize known ACPX, Codex, and OpenCode error fields. Preserve unknown messages. Redact known credential values and credential patterns before applying the 4,000-character limit. Do not copy response bodies, headers, stacks, or prompts. - Carry startup failures through the sidecar, Rust Runner, durable command receipts, and private transport error details. Keep provider text out of generic transport error strings and Sentry failure messages. - Keep turn failures in authenticated run events. Recover the same evidence from persisted events during server finalization. Store the cause in the company-scoped run result. - Show one shared notice on failed tasks and run details. Keep existing retry permissions, scheduled retry state, and resolved-task behavior. - Reapply registered run-secret and current-user redaction to task display data. Batch secret lookups for only the visible provider causes. - Document the contract and update the capability maps. Add regressions for nested Claude 429 errors, OpenAI quota and rate limits, OpenCode errors, redaction, durable receipt replay, restart finalization, and UI rendering. ## Verification - `pnpm -r typecheck` passed. `pnpm build` passed. All GitHub Actions build, test, package, and validation jobs passed on `d49c79ab4a`. The security checks passed. Greptile gave this head 5/5 with no open review threads. Paperclip Review remains pending. Its review worker exhausted six workspace recovery attempts with export_interrupted. A fresh review request was accepted and is waiting for that saved workspace. The ordinary Retry action rejected reuse of the stopped request. This is an external review-worker blocker; no check was overridden. The duplicate local `pnpm test:run` was stopped after the CI test shards passed. - Focused server settlement, provider evidence, and restart tests: 93 passed. Runner admission and normalization tests: 103 passed. Focused Runner wire and terminal tests: 29 passed. The real Rust/sidecar command receipt replay test passed. - The new projection test failed before implementation because the task had no provider cause. The startup producer regression also failed before extraction moved ahead of the generic error-code replacement. Both pass with this change. - Live check: a rebuilt local sidecar received an actual Claude HTTP 429 during startup. The new cause kept the full provider message. A synthetic task in an isolated local app then reached the same provider error through the native Runner path. The dedicated run result, task API, task banner, and run detail all showed the same message after reload. The later reporting fix keeps run.error generic; a regression proves the display cause still survives restart and stays out of Sentry capture. - The first isolated app attempt failed earlier in `admission_credential`. An ordinary manual Retry reached the provider and produced the 429 above. This PR does not claim to fix that separate admission failure. An initial parallel local database test setup also failed; the serial restart suite passed. - Security review found that copying display text into run.error could forward it to Sentry. A regression failed with that behavior. The fix keeps the cause in its display field. All 38 settlement/restart tests and the server typecheck pass. - All 29 protocol contract tests pass, including bounds and extra-field rejection. An earlier full Rust core run passed. The later full run hit one 5-second receipt-limit timing assertion; its isolated rerun passed. The full Linux and Windows Rust CI jobs passed on this head. - Review regressions pass for explicit Codex driver identity, native Rust startup and resume receipt propagation, oversized OpenCode/ACPX errors, task read-time registered-secret redaction, and the standalone source closure. The focused Runner run passed 103 tests; projection/restart tests passed 16. - Read-time redaction edge cases passed: a missing decryption key omits only the unsafe optional explanation; expanded redacted messages remain visible within the Unicode character limit. All 14 task projection tests and server typecheck pass. - The package Build check found an old Pi assertion against Error.message. The probe now checks the exact closed classification in the authenticated receipt and requires the new provider cause. Both polling transport cases pass. A local credential-free Pi probe stopped at admission_installation because this checkout lacks the Pi distribution; the installed Linux package probe passed in CI on this head. - UI coverage includes startup and turn text, unknown categories, literal rendering of provider text, read-only retry behavior, and resolved-task suppression. Token gates and map checks passed. - The live check covers Claude startup on macOS. Codex, OpenCode, terminal failures, and restart paths have deterministic regression coverage. The change is not deployed. ## Risks - Provider messages are untrusted text. The boundary selects specific fields, redacts credential patterns and known environment secret values, bounds the message, and renders plain text. New provider-specific secret formats may need added redaction patterns. - Optional fields preserve compatibility with older producers and stored runs. Old runs without the field keep their current fallback text. No database migration is required. - The cause stays in company-scoped run data. This change does not send provider text to first-party Telemetry or add it to OpenTelemetry span attributes. - Normalized categories and retryability are display evidence. Existing recovery classifiers, retry limits, ownership checks, and approval gates keep their authority. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and browser verification. The host does not expose the exact serving variant or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:13:26 -06:00
DottaandPaperclip	58308a3e63	perf(apps): scope sidebar review counts to one connection (#18688) ... ## Thinking Path > - Paperclip helps people manage AI agents and their governed app access. > - The Apps sidebar displays a Review badge for one connection. > - It polls the full company attention service every 30 seconds to get two counts. > - That service also loads every profile, rule and active catalog entry. > - This pull request adds an authorized connection read for only those counts. > - The same badge now avoids unrelated profile evaluation and database result decoding. ## Linked Issues or Issue Description **Current behavior** The contextual sidebar polls company-wide attention. It needs only one connection's signed pending action count and whether any catalog entries are quarantined. **Proposed behavior** Both sidebar variants poll a connection-scoped count endpoint with existing connection read authority. The full attention API stays unchanged. **Reason and benefit** The retained 200-profile fixture performs seven service queries and returns 80,115 rows for this badge. The new reader performs one query and returns one row. This reduces local service CPU and avoids unnecessary decoded profile/rule data. Related merged evaluator fixes are already in the baseline: Refs #18673, Refs #18681. A search found no existing PR for this scoped count reader. ROADMAP.md contains no duplicate planned work. ## What Changed - Add the narrow shared DTO, typed client, guarded REST/OpenAPI route and catalog-owned reader. - Count independent aggregates with both company boundaries and exact signed-argument semantics. Keep archived counts at zero. - Update both contextual sidebar variants. Keep the exact badge, 30 second interval and existing attention-prefix invalidation. - Add PostgreSQL differential tests, authenticated HTTP tests, sidebar tests, a repeatable benchmark and access documentation. ## Verification - Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, including both merged evaluator fixes. Service treatment: `0827f7d0b3`; route/fixture refinement: `230c42ee15`; final local existence gate: `390a218254`. - Passed: focused PostgreSQL differential test; assembled HTTP test with board session and agent JWT; 14 sidebar component tests; token gates; map checks; diff check. - Fixture: 200 profiles, 76,000 rules, 1,900 active tools, 695 removed references, 12 agents, a sibling connection and 2,000 unrelated historical invocations. Complete normalized two-count results are equal on both paths. - Quiet local measurement: seven alternating warm pairs after two warmup pairs, Node 24.22.2, macOS, real disposable PostgreSQL. Legacy/new service CPU median: 213.891/1.680 ms; ranges: 209.329–228.552/0.515–6.137 ms. Wall median: 156.867/1.248 ms; ranges: 151.739–162.289/1.115–1.386 ms. Queries: 7/1. Returned rows: 80,115/1. - Run `cd server && node --expose-gc --import ./node_modules/tsx/dist/loader.mjs scripts/benchmark-connection-review-counts.ts`. See `doc/connection-review-counts.md` for scope and failure history. Byte-volume accounting runs outside timing; no HTTP payload reduction is claimed. - Initial raw scalar SQL failed with ambiguous columns. Typed subqueries fixed it and passed real PostgreSQL equality. Initial fixture agents lacked permissions and correctly received 403. The final fixture grants explicit authority and tests revocation. An initial benchmark included byte serialization in timing; the reported run excludes it. - Passed required local `pnpm -r typecheck` and `pnpm build`. The initial sandbox typecheck could not open the tsx IPC socket; the escalated rerun passed. Full monolithic `pnpm test:run` was not run locally: repeating a 15–25 minute suite serially across both performance PRs adds no distinct evidence. Broad suite acceptance uses final-head CI, which runs the same general/serialized test partitions and the native Runner lane. Final-head CI is in progress. - Passed built-static Chromium journey on an owned local-trusted server and disposable PostgreSQL: Review 3 appears, only scoped counts are requested by this sidebar, resolving requests/clearing quarantines then reloading removes the badge, and archived counts are zero. The first ad hoc launcher omitted required decision-service options and failed at startup; using the normal no-op wake callback fixed that harness. - Greptile identified an invalid resolved-request fixture label. Corrected `denied` to the declared `rejected`, then reran the real PostgreSQL differential test. The count SQL and non-pending exclusion are unchanged. - The first final-head CI failed the static cross-tenant existence-oracle guard because the new explicit resource company assertion had no nearby `hasCompanyAccess` gate. Existing management admission already rejects foreign resources with 404. The route now also follows the standard local two-step gate. The unchanged guard plus authenticated HTTP negative cases passed (3 tests). No allowlist or test was weakened. - These local service measurements do not prove hosted performance. No deployment is included. ## Risks Low risk. New read admission reuses existing current-authority checks. The old attention API is unchanged. Counts still require database work over relevant pending/catalog rows. No cache, schema migration or telemetry is added. Roll back this PR if the badge differs or scoped reads deny previously permitted users. ## Model Used OpenAI `gpt-6.1-sol` (Sol 6.1), with reasoning, tool use and code execution. The host does not expose the configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 17:12:22 -06:00
DottaandPaperclip	554a770c6f	perf(apps): scope connection profile configuration reads (#18687) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Apps Permissions page reads saved access profiles for one connection. > - The page fetched every company profile, rule, and binding. > - It used only the app profile and eligible task-granted profiles for that connection. > - This pull request adds a saved configuration read owned by the profile service. > - The route keeps current connection and per-profile management checks. > - The benefit is fewer database rows, less Node work, and a smaller response for the same displayed configuration. ## Linked Issues or Issue Description **Subsystem affected** Server profile configuration reads and the Apps Permissions and Review data caller. **Problem or motivation** One connection page loaded and serialized unrelated company profiles. In the local fixture, this produced a 32.35 MB response with 76,001 rule rows. **Proposed solution** Select the displayed connection's app profile and exact eligible connection-intent profiles before loading their entries and bindings. Return an explicit configuration DTO with complete selected arrays. Keep summary and pending-review work on the existing full API. **Alternatives considered** Filtering the full response in the browser retains the database and Node cost. Truncating the full API would break other clients. Filtering entries to the connection would lose selectors and bindings needed by saves. **Roadmap alignment** This improves the existing governed Apps capability. It does not add a new access policy. Related merged work: #18673 and #18681. Related harness cleanup: #18683. Public PR searches found no duplicate connection configuration projection. ## What Changed - Add `ToolProfileConfiguration` and `GET /api/tool-connections/{connectionId}/profile-configuration`. - Select `app:<connectionId>` at any status and exact active, correctly bound `connection_intent` profiles. Run the existing per-profile visibility filter before configuration queries. - Preserve complete ordered entries and bindings. Keep all save behavior, empty-binding install fallback, and the existing full profile API. - Switch AppDetail and AgentConnectionAccess to the configuration DTO. Use a distinct cache key below the existing profiles prefix so mutation and reconnect invalidations still refresh it. - Add real PostgreSQL differential and authenticated HTTP tests, UI caller coverage, a reproducible benchmark, and access-map documentation. ## Verification Baseline: `3f8ebfff8aa9febd0733c6f65b07485dcadfd0e2`, after both #18673 and #18681. Final head: `b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`. Current implementation and reproduction details are in [the performance note](https://github.com/paperclipai/paperclip/blob/b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22/doc/performance/2026-11-02-connection-profile-configuration.md). - AppDetail: 79 tests passed. Existing access/action saves, install fallback, grant removal, errors, and refresh cases use the new caller. - Real PostgreSQL differential: passed. Full ordered selected configuration equals the old full-list selection. Cases cover extra bindings, sibling selectors, wrong source/key/status/binding, null and non-string agent metadata, fresh mutations, empty app bindings, visibility filtering, and company scope. - Production HTTP authentication: API key and run JWT cases passed. Hidden personal accounts omit spanning profiles. Private connection reads fail. Restored ownership restores visibility. Revoked management grants deny fresh calls. Final UID canonicalization and foreign-company 404 assertions passed on the current head. - Map structural check, token gate, and `git diff --check`: passed. - Required local `pnpm -r typecheck` and `pnpm build`: passed on the current head. `pnpm test:run` was started once. Its first chat shard passed (337 tests, 151.87 seconds); the second shard was intentionally interrupted after the complete final-head CI suite passed. This partial local attempt is not a full local test pass. The same stable-runner general and serialized partitions and Runner lanes passed in [final-head CI](https://github.com/paperclipai/paperclip/actions/runs/41538805231), including aggregate `ci / verify`. No local assertion failures were observed before interruption. - Final-head focused acceptance: four files, 85 tests passed. Full token gates, maps, and module boundaries passed. - Actual browser acceptance passed on the current checkout with isolated real PostgreSQL and a local trusted board. An action saved as Off persisted after reload. Both app agent bindings stayed intact. Removing an agent-only task grant persisted after reload. Requests used only the narrow profile endpoint. No browser exceptions occurred. No responses were mocked. No provider calls or hosted changes were made. The helper stopped the owned runtime and database after the check. - Final exact-head checks: 54 success, one clean neutral Paperclip Review, three skipped, zero pending or failures. Legacy Snyk status: success. No inline review comments. - Greptile reviewed exact head `b40c1ed7c0833d403e20ee8a5ff9d09f1bad6d22`: 5/5, no findings, completed successful check. Five alternating unprofiled pairs ran with real PostgreSQL on macOS/arm64, Node v24.22.2, and forced GC. The fixture has 210 profiles (200 heavy profiles plus ten eligibility cases), 76,001 rules, 1,900 active tools, 695 removed tools, and 12 agents. The selected configuration has two profiles and 1,391 rules. Each pair asserts complete ordered selected DTO equality. | Measurement | Full list | Configuration read | | --- | --- | --- | | Service wall ms, median [min, max] | 200.927 [193.319, 213.390] | 6.414 [5.508, 6.973] | | Node CPU ms, median [min, max] | 262.751 [246.727, 271.573] | 27.525 [22.576, 29.308] | | Select queries / returned rows | 7 / 78,139 | 3 / 1,397 | | Actual response JSON bytes | 32,353,534 | 592,233 | | Serialization wall ms, median [min, max] | 90.714 [88.715, 99.304] | 1.644 [1.637, 1.745] | | Serialization CPU ms, median [min, max] | 90.644 [88.569, 96.778] | 1.654 [1.637, 1.807] | Reproduce: `node --expose-gc --import ./server/node_modules/tsx/dist/loader.mjs server/scripts/benchmark-connection-profile-configuration.ts 5`. The benchmark measures the service and actual response JSON separately. It excludes HTTP and authorization work through an identity visibility callback. Node CPU does not include PostgreSQL CPU. No hosted or foreground-under-load improvement is claimed. Preserved failures: offline install lacked a locked tarball; frozen online install passed. Sandbox localhost and tsx IPC restrictions blocked initial DB/typecheck attempts; permitted retries use the same checks. The first HTTP suite needed the unbuilt plugin SDK; its owning build helper fixed setup. Shared typecheck caught a missing barrel export, now fixed. A broad typecheck inherited the host's unpinned nested pnpm executable; the owned stalled process was stopped and restarted with pinned pnpm 9. The complete current-head CI suite and aggregate verify passed. The separate Paperclip agent review published 5/5 with no findings at this head and completed with a clean neutral conclusion. An added foreign-company assertion expected 403; the existing guard correctly returned 404. Only the test expectation changed. The first real browser attempt saved and reloaded the action correctly, but grant deletion returned 422: the stress fixture had a company binding, which makes that profile the company default. The UI showed the existing deletion error. The second browser seed used the production agent-only intent shape and passed removal and reload. This does not change the stress fixture or product deletion gates. Initial external browser script attempts also needed an ESM extension and the correct dependency path. Logs, first-failure screenshots, traces, and result files are retained in the local evidence directory. ## Risks - SQL selection must stay aligned with connection-intent eligibility. Differential fixtures cover the exact current predicate. - A selected connection can still have many rules. This change preserves all selected configuration. - Per-profile authorization still reads current selected resources and can add work. Connection visibility does not bypass that check. - No migration, default-allow, effective-access, policy-review, provider, or deployment behavior changes. The old endpoint remains available. Rollback restores the old AppDetail caller and removes the additive endpoint. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and repository tools. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 16:55:56 -06:00
DottaandPaperclip	40b8075916	feat(ui): show scheduled task waits as Monitoring (#18624) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task indicators tell an operator whether work is active, waiting, or complete. > - An agent can yield until a saved monitor checks CI or another external result. > - The task currently keeps the In Progress icon between those runs. > - Changing the stored status would affect monitor admission and recovery. > - This pull request adds a read-only Monitoring display state with a blue clock. > - Operators can see and filter scheduled waits while execution keeps its existing rules. ## Linked Issues or Issue Description **What existing behavior does this improve?** Task status indicators, task collection filters, board columns, and search facets. **Current behavior** A task that waits for a scheduled monitor shows the same In Progress icon as active work. **Proposed behavior** Show Monitoring with a static blue clock when an agent-owned task has a saved ordinary monitor and no active run, execution lock, review participant, or active recovery action. Return to the normal task indicator when the check starts. Preserve review, blocked, terminal, and quota-recovery states. **Reason and benefit** Operators can distinguish scheduled waits from current agent work without reading the task history. **Breaking changes** None to stored task statuses or execution rules. Task reads gain optional displayStatus. Collections and search gain a separate read filter. Existing status filters keep their lifecycle meaning. Related historical work: #4396 covered a monitoring label exemption. This change uses the saved monitor and a read projection. ## What Changed - Add one SQL projection for task detail, compact lists, summaries, relations, search results, and display-status counts. - Show the blue clock in shared task icons and chips, task detail and breadcrumbs, properties, references, list rows, recent navigation, and agent task cards. - Add Monitoring to task filtering, grouping, sorting, board columns, and search facets. Prevent a drag from creating this derived state. - Keep stored lifecycle statuses, monitor scheduling, checkout, recovery, permissions, and agent status filters unchanged. - Add database and UI regression coverage, full-page Storybook examples, and product and access-map documentation. ## Verification - Passed `pnpm -r typecheck`. - Passed `pnpm build`. - Passed `pnpm check:token-gates` and the map/catalog structural checks. - Ran the new database suite on a disposable PostgreSQL 17 database. It covers read surfaces, filter pagination/counts, queued/running checks, rescheduling, completion, retained monitors, review, and recovery holds. - Ran existing search, list, board, sidebar, task status, and filter tests. Corrected board queries use displayStatus and eight columns. - The full local `pnpm test:run` stopped in a chat integration shard: 12 failures (four timeouts and eight FIFO publication assertions). That run spanned a rebase. A focused rerun on the current source passed all 21 selected cases without assertion or timeout changes. The original cause is not established; this does not claim the full local suite passed. Final-head required CI passed (ci / verify and ci / e2e, run 41488267209). - Passed 82 tests for task API summaries, recovery precedence, search, and optimistic updates; 54 tests for status cards, database reads, and board moves; and 27 tests for blocker relations and recent-task cache updates. One intermediate database setup hit its 20-second hook timeout; the serial rerun passed. - Greptile is 5/5 on d003ea672d. All review threads are resolved. The separate optional Paperclip Review ended early and returned neutral; GitHub rejected a direct rerun request with HTTP 404. It is not counted as a completed review. - Review fixes cover escalated recovery, is:open, optimistic status changes, blocker-summary projection, and recent-task dedupe. - Reviewed 15 screenshots from production pages and components with isolated Storybook API fixtures: task detail/properties/breadcrumb, list, groups, status filter, board, search/results/filter counts, reference pills, sibling navigation, agent cards, picker, dashboard, inbox, profile, watched status-card tasks, light theme, and mobile. This does not claim deployed staging verification. ## Risks - Read projections must stay consistent across full, compact, summary, and search responses. Tests cover those paths. - Monitoring is not a writable lifecycle status. The board does not accept drops into its Monitoring lane or onto a monitored card. This prevents a drag from implying a scheduled check. Move work into In Progress through that lane instead. - The read projection adds run/recovery existence checks. Existing heartbeat_runs_company_issue_created_idx and issue_recovery_actions_company_source_status_idx indexes cover the company/task lookup. It adds no worker, timer, recurring query, migration, or provider call. No production load benchmark is claimed. - Older clients can ignore displayStatus. Existing lifecycle status filters keep their behavior. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository tools, code execution, and browser tools. The exact backend model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 11:35:30 -06:00
DottaandPaperclip	5e4cc01073	feat(workspaces): keep remote files off the webserver disk (#18403) ... Keep native sandbox workspace files independent of the webserver disk. Use encrypted saved versions and Postgres ownership so a replacement server can recover files and continue work. Enable the path by default for new eligible workspaces when durable storage is configured. Preserve explicit opt-outs and existing host workspaces. Include safe snapshot repair, Files access, import, migration and retention. Bound saved-Files resources, preserve imported Git recovery, reject unfinished Git operations, publish immutable concurrent captures, and verify permanent object-version deletion. Validated 471 targeted local controls, build/typechecks, all 40 CI partitions, fresh Greptile 5/5 and security review. The PR records original failed attempts and scoped retries. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-31 14:54:01 -05:00
DottaandPaperclip	8ff812cdde	feat(chat): deliver proactive task messages to agent chats (#18374) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat gives each person a persistent conversation with each agent. > - Tasks and routines produce results outside that conversation. > - Those runs need a safe way to bring a requested result back to the person. > - The server must select the agent's own chat and check the source task's access rules. > - This pull request adds proactive chat messages and unread indicators. > - People can see the result, open its source task, and reply in the same chat. ## Linked Issues or Issue Description **Subsystem affected** Server, native runner tools, shared contracts, database, and UI. **Problem or motivation** An agent can report progress on a task or comment on an authorized task. It cannot send a routine result to its own standing chat with the responsible person. A morning review routine therefore cannot leave a visible digest in that person's chat. **Proposed solution** Add `send_chat_message({body, idempotencyKey})` and `POST /api/agents/me/chat/messages`. The server selects the company, agent, and responsible person from the active run. It saves an attributed comment and audit record in one transaction. The chat shows a source-task link and per-person unread state. **Alternatives considered** A destination parameter on a generic comment tool would expose unnecessary routing authority. A new delivery worker would add another durable delivery path. This change uses a dedicated tool and the existing comment and live-event paths. **Roadmap alignment** This extends the experimental Agent Chat capability in `ROADMAP.md`. The existing `enableAgentChat` setting controls it. Related public work: Refs #13282, Refs #13283, and Refs #14408. Searches for proactive chat and unread chat found no duplicate PR. ## What Changed - Add a focused messaging service shared by the native tool and HTTP route. It checks current run ownership, execution mode, recipient membership, task access, and task privacy. - Reuse the canonical company/agent/person conversation. Save the comment and audit record together. Keep task completion and chat execution separate from delivery. - Add source-task provenance and source-scoped retry keys. Identical retries return the saved receipt. Changed content returns a conflict. - Enforce the conversation's fixed agent at the shared comment-write boundary. Keep normal reply cancellation and session checks. - Add chat unread state, accessible agent dots, aggregate Chat indicators, and visible-message read cutoffs. Refresh through existing live events. - Add the request-key constraint, native tool contracts, documentation, capability maps, stories, and regression coverage. ## Verification - Local delivery/security proof: all 38 proactive-message cases pass. Eight credential-scope regressions and four quota regressions fail against the prior source. The repaired delivery and canonical cross-issue cap suites pass all 49 cases; all 31 OpenAPI tests, server typecheck, and capability-map checks pass. The shared per-run cap covers native and HTTP sends, mixed ordinary mutations, concurrent last-slot contention, and retry/recovery without double charging. - Credential tests preserve the authenticated person and authorizer; reject bridge/test credentials and persisted skill-test runs after task-mode changes; and recheck API-key revocation, scope, and responsible person after destination locking. The earlier delivery/native-authority suites passed all 74 cases before the four quota additions. - Four PostgreSQL contention cases cover native and HTTP sends plus retries during a native chat mutation. All four deadlock against the earlier source and pass with destination-before-source/run locking. Recovery and human-reply delivery enqueue only the human comment; a proactive digest never wakes the chat agent. - Passed 108 companion conversation, native-authority, and OpenAPI tests; 112 runner catalog/workflow-report tests; 47 completion-delivery tests; and 25 project-handoff tests. Focused runner tests cover native tool availability without generic API tools. - All 327 affected UI tests, UI typecheck, token gates, and catalog checks pass. The read hook retries at most three times, respects server retry delay, coalesces only seen cutoffs, recovers on reconnect, and cancels on scope changes. - Real browser journeys with a deterministic process agent verify routine-to-chat delivery, first-chat creation, unread state across reload, the source link, secondary chat navigation, and a follow-up reply referring to the digest. This does not qualify live GitHub retrieval or model output quality. - The Storybook showcase uses production components with deterministic network fixtures. Rendered interactions pass for the landing, ordinary read success, transient read recovery, and persistent failure after bounded retries. Both Chat navigation variants update from two unread chats to one while only Kodi's dot clears. Mobile and light variants were reviewed. The existing Coordinator Recents label contrast finding is unchanged; no Linux visual baseline archive exists. - Current master's independent sandbox migration retains 0418. Regenerated the feature migration as 0419 from the combined schema and retained exactly the newest five snapshots; numbering/safety, replay, and schema-drift checks pass. - Workspace typecheck and build passed before PR preparation; canonical server/UI typechecks passed again on the combined tree. Migration `0419_slippery_impossible_man.sql` passes safety and schema drift checks on disposable PostgreSQL, including interrupted index repair, concurrent migrators, one-time owner read-state backfill, preserved receipts, and two SQL replays. A default five-second local snapshot comparison timed out; a diagnostic 30-second run passed without drift. - Earlier failures are retained in the record: full local `pnpm test:run` and broad shards encountered embedded-PostgreSQL startup, timing, and provider-recovery failures. Thirteen selected external-channel cases pass in isolation on the original and changed sources; an unordered comment-row assertion now explicitly orders both reads. The earlier route registry, native tool count, runner catalog, and accessible Chat badge-name failures were fixed and passed full remote CI on `920b3bb7db`. - Repeated managed-skill server8 failures came from CI's newer synthetic merge: its Cloud guard correctly rejected the old local-disk fixture. Master independently repaired the fixture in #18385; this branch keeps that repair unchanged. All 57 managed-skill and Cloud prerequisite cases pass on the combined tree, and that fixture has no remaining PR diff. - Current reviewed head `dc16f314739a6d977bed21cb1446d0e80a541578`: Greptile 5/5 with no findings, Superagent security assessment passed, advisory Paperclip Review 5/5 with no findings, and zero open review threads. All 55 current-head check runs pass, with five expected skips and the advisory review neutral at 5/5; Snyk also passes. No checks are pending or failing. Protected squash auto-merge is enabled and is waiting for required code-owner approval. A normal exact-head merge was refused by the base-branch policy; no admin bypass was used. The previous head's Superagent quota finding is fixed and covered above. ## Risks - Chat remains company-visible. Delivery rejects a source task with private-task, project, or ancestor restrictions. - Read state uses a comment timestamp cutoff. The server retains database precision and never moves a read receipt backward. - The new unique constraint changes comment query planning. An existing regression now uses an explicit row order. The migration builds the unique index concurrently, then takes a brief table lock to attach it. It uses the existing top-level migration and interrupted-index recovery path. - The upgrade backfills missing chat-owner read-state rows at the latest existing agent comment. Existing read receipts remain intact. New chats can still be unread before the human participates. - Quota observation commits atomically with delivery. A rejected send rolls back its attempted denial audit row and any first-chat creation; the canonical warning remains in server logs, and previously spent quota remains enforced. Identical/conflicting retries and receipt recovery spend no extra slot. - This version delivers in-app only. It adds no external-channel, push, or email delivery. - Local full-suite acceptance remains limited by the failures recorded above. The feature stays behind `enableAgentChat`. ## Model Used OpenAI Codex. PR preparation and CI work used `gpt-6.1-sol` with tool use and code execution. The earlier implementation also used Codex, but its exact model ID was not recorded in the visible session. The context-window size was not provided. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run focused tests locally and they pass; the broad local failures are recorded above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 21:39:07 -05:00
DottaandPaperclip	aa58fa1073	perf(ui): restore fast Coordinator task and inbox loading (#18323) ... ## Thinking Path > - Paperclip lets people manage work done by agents. > - Tasks, Inbox, and Needs You must stay usable as work history grows. > - The Coordinator views bypassed some existing collection optimizations. > - Navigation fetched complete decision cards for a count, and Tasks mounted every loaded row. > - This PR uses the canonical attention rules for small count and task-ID responses. It also restores bounded rendering and the optimized Inbox selection. > - The benefit is less data and less initial rendering work without a second definition of pending work. ## Linked Issues or Issue Description **What happened?** Coordinator navigation fetched the full attention feed on every cold collection visit. Tasks rendered the entire loaded collection. Coordinator could also select the old Inbox loader when Streamlined UI was off. **Expected behavior** Fetch only the projection needed by each surface. Start with a bounded set of rows. Keep complete task membership, accurate counts, current permissions, and stable loaded rows. **Steps to reproduce** 1. Enable Coordinator and open Tasks or Inbox with an empty browser query cache. 2. Observe a complete attention-card download for the navigation badge. 3. Open a large task list and observe that every loaded row mounts at once. 4. Disable Streamlined UI while keeping Coordinator enabled, then open Mine and observe three overlapping compact task reads. Related work: #17756, #3744, #17983, and #18312. ## What Changed - Add board-only attention count and complete task-ID projections. Reuse canonical company, privacy, addressee, deduplication, dismissal, snooze, and archive rules. Skip card bodies, plan/image reads, and presentation roster lookup. - Share the small count read across desktop and mobile navigation. Reuse the page-owned response on Needs You and Needs attention. Keep the newest known count through navigation, including zero. - Use the task-ID index for Needs attention and keep task row hydration in batches of at most 100. - Render 100 task rows initially, across the whole expanded tree. Reveal 150 more at a time through scrolling or a button. Keep existing rows through refresh and append. Defer offscreen leaf layout with shared tokens. - Keep Coordinator on the optimized Inbox when Streamlined UI is off. Use index reads for expanded project recents. - Add real-database, component, and production-build browser regression checks. Update the access and feature maps. ## Verification - All required CI checks pass on `1c68ac8957490a6c8585ba0939355f536c8120bf`: typecheck, build, server/workspace tests, serialized database suites, and eight E2E shards. Greptile is **5/5**, with its row-retention finding fixed and the review thread resolved. Storybook checks were intentionally skipped; Paperclip Review completed neutral. - Workspace typecheck and production build passed locally. The final rebased source also passed UI typecheck and production build. Token gates and map checks passed. - The attention service suite passed all 34 tests on disposable PostgreSQL. Final targeted checks cover compact projections, complete-set counts, quality approvals, snoozes, private tasks, and board/company access. Database response capture confirms compact projections do not read plans or images. All 31 OpenAPI tests passed after registering both endpoint contracts. - Six production-build browser journeys passed: desktop, mobile, Coordinator Inbox with Streamlined UI off, Needs You, Needs attention, and an older server page followed by task open/Back. They verify the initial 100-row budget, scrolling, retained DOM identity/height, one owner per attention read, complete task membership, 220 retained rows on Back, and less than 2 px of return-position difference. - The row-budget and Inbox regressions fail against original source and pass with this change. The final three affected UI suites passed 40 tests, and the broader Inbox/IssuesList suites passed 101 tests serially. ### Controlled production-build comparison Baseline `b31ec0a2dc079e32272a8f618425fe92b68bbaec` versus PR head `1c68ac8957490a6c8585ba0939355f536c8120bf`. Isolated baseline asset `index-CS9uPxMx.js`; candidate asset `index-BxJeTQ30.js`. Both UI builds used the same warm disposable API server and database, with 220 tasks and 110 pending board approvals. Chromium **151.0.7922.34**, 1440 × 900 viewport, **4× CPU throttle**, loopback with no added network latency, alternating baseline/candidate order. Cold cases use a new browser context with network cache disabled; warm cases navigate within the same context. Other host workloads are uncontrolled. Readiness requires the destination collection's first visible task row, then two animation frames. Inbox readiness waits for its canonical Mine route so previous-page rows cannot satisfy the measurement. Five paired samples per case; cold Inbox has a second five-pair confirmation wave because the first result was slower. Times below are **milliseconds**; parentheses show min–max. Paired savings are the median of individual before-minus-after pairs, not the difference between medians. | Journey | Pairs | Baseline median (range) | Candidate median (range) | Median paired saving | Faster pairs | | --- | ---: | ---: | ---: | ---: | ---: | | Cold Tasks | 5 | 2752 (2542–3025) | 2300 (2275–2524) | 328 | 5/5 | | Cold Inbox | 10 | 2536 (2283–2927) | 2582 (2117–2856) | 37 | 5/10 | | Warm Tasks navigation | 5 | 2054 (1826–2479) | 1448 (1419–1548) | 635 | 5/5 | | Warm Inbox navigation | 5 | 2020 (1485–2254) | 1568 (1289–1848) | 350 | 5/5 | | Task → Back to Tasks | 5 | 1747 (1609–1948) | 1327 (1238–1355) | 472 | 5/5 | Cold Tasks, warm Tasks, warm Inbox, and Back improved in every measured pair. **Cold Inbox latency is inconclusive:** the two waves pointed in opposite directions, with the candidate faster in 5/10 pairs. This PR claims less badge work there, not a proven cold Inbox latency improvement. The canonical badge response shrank from **200,039 to 359 decoded bytes** (encoded HTTP transfer: **8,125 to 598 bytes**). The compact response also skips card payloads, plans, images, and presentation-only roster work. These are measured mechanism reductions, separate from end-to-end latency. Needs You still owns its full card feed; Needs attention owns its complete task-ID projection. Reproduction setup and assertions are in `tests/perf/collections/README.md`. <details> <summary>Individual paired readiness measurements (ms)</summary> | Journey | Pair | Baseline | Candidate | | --- | ---: | ---: | ---: | | Cold Tasks | 1 | 3025 | 2283 | | Cold Tasks | 2 | 2542 | 2275 | | Cold Tasks | 3 | 2752 | 2424 | | Cold Tasks | 4 | 2783 | 2300 | | Cold Tasks | 5 | 2641 | 2524 | | Cold Inbox | 1 | 2927 | 2702 | | Cold Inbox | 2 | 2422 | 2629 | | Cold Inbox | 3 | 2409 | 2606 | | Cold Inbox | 4 | 2621 | 2769 | | Cold Inbox | 5 | 2539 | 2856 | | Cold Inbox | 6 | 2283 | 2117 | | Cold Inbox | 7 | 2466 | 2558 | | Cold Inbox | 8 | 2674 | 2286 | | Cold Inbox | 9 | 2696 | 2389 | | Cold Inbox | 10 | 2533 | 2183 | | Warm Tasks navigation | 1 | 2479 | 1423 | | Warm Tasks navigation | 2 | 2228 | 1548 | | Warm Tasks navigation | 3 | 2054 | 1419 | | Warm Tasks navigation | 4 | 1826 | 1448 | | Warm Tasks navigation | 5 | 1954 | 1518 | | Warm Inbox navigation | 1 | 2020 | 1661 | | Warm Inbox navigation | 2 | 2198 | 1848 | | Warm Inbox navigation | 3 | 1733 | 1497 | | Warm Inbox navigation | 4 | 2254 | 1568 | | Warm Inbox navigation | 5 | 1485 | 1289 | | Task → Back to Tasks | 1 | 1721 | 1330 | | Task → Back to Tasks | 2 | 1747 | 1275 | | Task → Back to Tasks | 3 | 1948 | 1355 | | Task → Back to Tasks | 4 | 1609 | 1327 | | Task → Back to Tasks | 5 | 1790 | 1238 | </details> ### Failed attempts and limits - The local full repository command hit embedded PostgreSQL startup failures and 16 failures in the untouched native session executor suite; it was stopped. The full local suite is not claimed green. CI on the final revision provides the broad test result. - A concurrent local UI run hit timing failures under host contention. Inbox/IssuesList subsequently passed all 101 tests serially. No tests were weakened to address these timeouts. - CI caught two integration gaps introduced during this work: missing OpenAPI entries for the new endpoints, and an attention fixture incompatible with the routine filters added on master. Both were fixed; the affected 31 OpenAPI and 40 UI tests pass. - The final revision's first CI attempt failed two untouched tests: scheduled-retry batch cancellation and the tool-access audience/archive race. Both failed jobs passed when rerun without changing the revision. Their original failed attempts remain visible in [the workflow history](https://github.com/paperclipai/paperclip/actions/runs/41083379875). - Browser evidence uses synthetic disposable data. This PR is not deployed staging or production verification. ## Risks - New attention sources must preserve canonical eligibility. Both projections call the existing collector, so they share its permissions and complete-set behavior. - The count and task-ID caches must not stand in for full decision cards. Each projection has its own key under the existing attention invalidation prefix. - A row budget can omit descendants if applied per parent. The tests cover one global budget across a large expanded tree, refresh, append, reset, and incremental loading. - Local browser evidence uses synthetic data and is not staging deployment proof. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use, and code execution. The exact backend model deployment ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (affected suites; broad-run limitations are listed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 11:58:53 -05:00
DottaandPaperclip	f1911a4c53	perf(skills): move repository packages out of Postgres (#18082) ... <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/205/changes/paperclip/205 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents receive selected company skills before each turn. > - Imported skills can need files from the whole source repository. > - The current path stores those files in Postgres and reads large JSON payloads during preparation. > - This pull request stores verified packages in the configured file or object store and keeps preparation queries small. > - It also converts old rows automatically and restores missing files from the exact saved commit. > - The result reduces database traffic while preserving skill versions and repository dependencies. ## Linked Issues or Issue Description **What existing behavior does this improve?** Company skill import, version storage, and preparation before agent turns. **Current behavior** Repository snapshot bodies live in JSONB. Skill preparation can read those bodies again even when the files already exist on disk. Historical packages and native assets also need a bounded cleanup path. **Proposed behavior** Store package archives and manifests through the existing storage service. Keep version identities, receipts, hashes, and recovery metadata in Postgres. Convert legacy rows after the stored package passes read-back verification. A missing package can use legacy bytes or its exact Git commit under current repository authorization. No separate data-migration command is required. **Reason and benefit** The warm path sends no repository bodies through SQL and reads no object bytes. Cleanup runs through the existing deadline coordinator. Current versions, active runs, explicit pins, and authored content retain their protection. **Breaking changes** The normal application schema upgrade is required. Single-version and skill-file responses keep their existing shape. Version lists, current-version summaries, and test-run version summaries now contain file names and kinds without file bodies. Diff and restore fetch selected versions through the existing single-version endpoint. Older server builds must not write after conversion unless they support the new package format. The storage override and rollback limits are documented in `doc/skill-package-storage.md`. **Additional context** Related work: #15731 preserves repository dependencies; #17923 changes version-cache scheduling; #18080 changes delivery to remote Codex homes. This change owns package persistence, acquisition, and expiry. It does not replace those delivery or scheduler changes. ## What Changed - Add company-scoped archive receipts, exact-commit recovery recipes, migration deadlines, and a durable object-deletion queue. - Add bounded archive upload, read-back verification, extraction, concurrent request coalescing, and verified local caches. - Convert old snapshots and versions automatically. Keep legacy bytes when verification fails. - Recover missing files through current source authorization and the original commit. Validate repository identity, file hashes, modes, and the aggregate hash. - Expire unused recoverable Git packages and native assets after 30 days. Preserve active pins and the durable copy of authored or unproven content. - Keep warm preparation queries metadata-only. Bound file digest caches and avoid repeated body hashing when file identities are unchanged. - Extend the existing storage contract with cancellation. Register package work with the existing coordinator. Optional conversion retries retain the original bytes and may wait for ingress without blocking sleep. - Add regression tests, operating guidance, a deep trace, benchmark receipts, and an optional maintenance command. ## Verification ### Final master integration and review repair — October 29, 2026 Current head: `a6065424249e763dfeac37161f1b52a9f25bd9b7`. Commit `f8bbdd3dbd` resolves the conflicts with master `76a796e964`; the final two commits repair the Apex deletion findings below. - Keep both package-maintenance and version-retention notifications in their write transactions. - Keep the shared run-pin predicate and master's controller-lease deadline metadata. Package expiry reads the updated protected-version result. - Preserve every master migration entry. Generate migration **0415 (`0415_zippy_zemo.sql`)** after master's 0414. Its skill SQL is byte-for-byte unchanged from the previously tested idempotent migration. Retain only the five newest schema snapshots. Migration numbering and safety checks pass. - Repair failed file/folder deletion: preserve a complete temporary subtree until transaction commit, restore on upload or SQL failure, and restore callback failures before releasing the mutation lock. Both PostgreSQL regressions reproduced missing files before the fix and now pass. They cover binary data, executable permissions, empty folders, unlisted files, unchanged database/version rows, successful retry after upload-claim expiry, and historical reads. - Reconcile lost commit replies under the same advisory and row locks. Check the saved version and source identity before restoring. Keep committed deletions intact and preserve backups when the database outcome is unknown or a newer edit exists. Four real-PostgreSQL outcome tests pass; the prior code failed three of them. - Full workspace typecheck and production build pass on the final commit. All **204** focused PostgreSQL package/cache/retention/authoring and filesystem-boundary tests pass, with none skipped. - Apex review **18 passes 5/5**, with zero new comments on the exact final head. All **34** review conversations are resolved. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/40914116508) completes successfully on attempt 2. The head has **55 successful checks**, two neutral reports, two skipped checks, and a successful Snyk status. No checks are pending or failing. All eight browser shards pass. - Runner Vitest 2/2 timed out once in the unchanged ACPX controller-replacement fixture. The same job passed on the preceding commit. One unchanged CI retry passes all five ACPX recovery cases, **2,253** Runner package cases and **126** native server integration cases; their standard one and five skips remain. The initial timeout's root cause is not established. Local diagnosis also hit the fixture's eight-second session.open timeout with an authenticated Runner; no local Runner pass is claimed. - Both full local `pnpm test:run` attempts were stopped when Apex identified deletion defects and source repair became necessary; neither is claimed as a passing full local run. The final **204** focused tests, full workspace typecheck, and production build pass. Full-suite acceptance uses the final-head CI jobs. The superseded merge-head CI included runners terminated by shutdown signals; each source repair started a new full run. - Auto-merge is enabled for squash merge. The final head is conflict-free and all checks pass. GitHub's active master ruleset still requires a code-owner approval for the dependency files. The existing review requests are with `devinfoley`, `nickyleach`, and `forgottendev`; no policy bypass was used. - The earlier hosted stress and full native receipts below remain tied to `a2e597d800`; they are not relabeled as a deployment of this final integration. The earlier local PostgreSQL startup failure remains recorded below; PostgreSQL ran successfully for this integration verification. ### Full native skill execution — October 29, 2026 The full native-agent acceptance gap is closed on candidate `a2e597d800eb989d565780e7aa947d5368676444`. Two qualifying native turns used OpenAI API / `gpt-5.4-mini` in real Daytona sandboxes, on the same disposable hosted TEST tenant as the stress test. A follow-up on the same task consumed a newly saved skill version. The task text asked for the assigned skill workflow. It did not contain the proof nonce or expected receipt. The assigned authored skill contained a Python verifier and a random package-local nonce. The agent read the actual installed skill files and executed that verifier against the imported, automatically converted 1,360-file repository package. It checked the repository file count and reference/binary bytes, and directly executed the repository's executable probe. Independent verification matched the emitted JSON to the nonce and expected byte hashes. The agent persisted the same receipt as its task response; native finalization committed, workspace finalization succeeded, and task state became Done. | Qualifying turn | End-to-end run time | Recorded skills.prepare | Repository files | Binary hex | Shell receipt SHA-256 | |---|---:|---:|---:|---|---| | Initial task | 47.881 s | 1822 ms | 1360 | `00ff07800a` | `a1deb30dc095f2306c7aa9cc5ec396da12c68961cf5b809f4d7c20f7fc68493b` | | Corrected follow-up, new nonce/version | 37.974 s | 377 ms | 1360 | `00ff07800a` | `dbcdc33fa1376b4edc136feae1cdf770560a73292467e93619efdda20e4282a7` | Both successful shell receipts reported `probe: STRESS18082` and `assetSha256: e8ea7a2f15f7bb3b332a46adbef848465884c8996595aa234748befd544d1f3a`. Their proof nonces, selected version IDs and receipt hashes differed, and both ran on the same task. These are observed timings, not latency SLOs; they include real provider work. The service stress-test timings remain a separate measurement boundary. Earlier failures remain in the evidence: - The original five-minute Daytona allocation timeout never reached model execution. After normal owned-stack restore, its durable cleanup record was `expired` / `success`. The cause of that original allocation timeout remains unknown; a later success establishes recovery only. - Initial continuation setup used unsupported generic agent-update fields and returned HTTP 409 before dispatch. Setup was corrected to use the supported skill-assignment and versioned instruction commands. - The intervening completed native turn read repository bytes and ran the probe, but omitted the verifier JSON under the older test agent's overly-specific instruction file. Its strict receipt validator failed; it is not counted as a qualifying turn. Correcting the test agent's instruction file and sending an explicit task-scoped wake produced the qualifying follow-up above. - The pinned application's immutable rate card omits `gpt-5.4-mini`, so completed usage triggered the normal block-on-unpriced budget policy. Supported append-only accounting corrections estimated the three completed runs using the [official OpenAI model rate card](https://developers.openai.com/api/docs/models/gpt-5.4-mini): $0.75 input, $0.075 cached input, $4.50 output per million tokens, assuming standard processing. Estimates totaled 12.482655 cents. Original usage and missing-price records are preserved; the $1 hard stop and unpriced-usage guard stayed active. - Model shell attempts included a missing `xxd` command and incorrect guessed file paths. Those failed calls and subsequent successful actual verifier calls remain visible in complete event streams. No zero-tool-error claim is made. No product source changed during this continuation. The previously measured contention, cold-start and memory limits remain. The later merge resolution is covered by the separate integration checks above; these hosted receipts qualify their recorded candidate. ### Hosted stress test — October 29, 2026 **Hosted acceptance:** full native-agent execution and a changed-skill follow-up pass on the recorded hosted candidate. The final integration section above records the subsequent conflict resolution and its separate checks. Tested baseline `929e840595c8933ca485ac7a2e99413e1e11ca64` and this exact candidate sequentially on one disposable TEST tenant, with external Neon PostgreSQL and actual S3. The fixture contained **1,360 files / 22.4 MB / ten skills sharing one repository**. Measurements cover deployed skill-preparation and native-context services in an adjacent diagnostic process, not a complete agent turn. The container had **24 vCPU / 24 GB**. | Warm workload | Median before → after | p95 before → after | DB returned value bytes/preparation before → after | |---|---:|---:|---:| | One skill, serial | 1,060 → 125 ms | 1,303 → 155 ms | 22,610,389 → 1,771 | | One skill, four concurrent | 4,415 → 297 ms | 5,143 → 328 ms | 22,610,389 → 1,771 | | Ten skills, serial | 5,723 → 535 ms | 6,057 → 600 ms | 224,739,030 → 10,671 | | Ten skills, four concurrent | 24,410 → 1,986 ms | 24,833 → 2,221 ms | 224,739,030 → 10,671 | - Candidate warm preparation performed **zero S3 GETs**. Ordinary-pool queries fell from 16 to 9 for one skill and 61 to 9 for ten, excluding dedicated lock connections. DB numbers measure returned values before Drizzle decoding, not wire bytes. Baseline/candidate sample counts were 30/100, 32/100, 5/50 and 8/48 respectively; the small baseline ten-skill samples are comparative evidence, not SLO estimates. - **Application-UID soak:** 800 ten-skill preparations over 10.3 minutes, four workers paced by 2.5 seconds; **zero failures**, median **559 ms**, p95 **775 ms**, p99 **910 ms**, zero warm S3 reads. All 726 accompanying health/company-list requests returned HTTP 200. Their end-to-end p95 was 569/502 ms, including network latency. The probe overlaps most of the soak. Soak DB-response capture was limited to its first 60 seconds. - **Important remaining costs:** an unpaced 16-way ten-skill burst had p95 **8.32 seconds** because company-level preparation serializes. Cold preparation took approximately **1.1–2.5 seconds** for one skill and **7.4–10.0 seconds** for ten. Ten-skill diagnostic RSS remained about **1.17 GiB** (baseline 1.18 GiB), and the soak peaked at **1.35 GiB**. A concurrent whole-container sample was 3.45 GB. This does not establish suitability for a small-memory host or eliminate per-entrypoint native asset duplication. - **Upgrade without a manual data migration passed:** legacy snapshot JSON shrank from 22,465,691 bytes to `[]`, with a stored package and zero pending conversions. - **Recovery passed:** sixteen concurrent cold requests coalesced into two S3 GETs; full bytes/binary/executable modes matched; same-size corruption repaired; an injected storage failure preserved warm access and failed cold access safely. A disabled public-Git source denied recovery, and after the actual five-minute backoff its exact SHA reconstructed all 229 files. Cross-company descriptor access was denied. Private-Git credential revocation was not tested. - **Retention passed:** reclaimed 27,754,496 allocated cache bytes, deleted an unreferenced package row and both objects, preserved aged authored history, and expired/reacquired reconstructible Git content. A version pinned by the failed native run was correctly retained. Only owned fixture timestamps were aged. - **Browser UI passed:** authored skill creation, saved edit surviving reload, version history, and restoration of earlier content. - **Initial native attempt (superseded by the qualifying turns above):** one real run failed at Daytona allocation after five minutes (`setup_failed`), before any model execution. Provider cleanup also reported failure. That attempt does not count as a passing native turn. Its receipt and all evidence remain preserved; its cleanup later recorded success after normal restoration. - Initial A/B commands ran as SSH root. Diagnostic-owned lock/cache permissions caused one setup HTTP 500 and the first UID-1000 soak to fail; exact fixture paths were repaired. The final soak, retention and recovery ran as application UID 1000. A fresh UID-1000 one-skill batch also passed (30 samples, median 116 ms, p95 321 ms, 1,771 DB value bytes/preparation). Earlier harness-export, SSH, npm provenance, backoff and UI-wait failures remain recorded alongside their successful retries. - Private evidence includes raw timings, assertions, HTTP probes, screenshots and failure/cleanup receipts. Initial failed-campaign cleanup suspended and retained the owned TEST tenant and released the shared QA account lease at **15:22:41 UTC**. The tenant was subsequently resumed under its existing ownership for the native execution qualification above. Final cleanup completed at **15:53:09 UTC**: the account lease is released and the tenant is suspended/retained under the original October 30, 12:24:52 UTC retention deadline. Its separate stack lease remains held; no active or pending-cleanup environment lease remained. The original retention window is deliberately preserved even after successful continuation. - Earlier hosted-tested head: `a2e597d800eb989d565780e7aa947d5368676444`, integrated with master `929e840595`. The earlier merge conflicts were resolved at this head. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/40884087629) passed; all checks are complete with 55 successes, two neutral reports, two skips, and no failures. [Apex review 15](https://github.com/paperclipai/paperclip/pull/18082#issuecomment-6480257258) passed 5/5 with zero findings on this exact head. All review conversations are resolved. - Ten real-filesystem boundary cases and server typecheck pass locally for the security fix. Coverage includes cache roots, descendants, individual files, symlink aliases, recursive ancestors, a future concurrent cache fill, and allowed sibling directories. [Current-head private-project integration tests](https://github.com/paperclipai/paperclip/actions/runs/40884087629/job/123483306405) passed all 29 cases, including authorized and unauthorized imports of a cached private package. Full typecheck, build, server suites, all eight browser shards, and Runner checks passed in current-head CI. Local database tests could not run because PostgreSQL could not initialize; this limitation is recorded below. - The earlier generated migration preserved the newest five snapshots. The final integration renumbers its unchanged SQL to 0415. Migration safety/numbering checks pass. Its SQL is byte-for-byte identical to the previously verified idempotent skill migration; repeated application on this latest schema was blocked by the same local database startup failure. - The broader verification before the final migration-only rebase passed 1,025 package, lifecycle, remote-launch, native-session, and asset-retention tests, 52 focused UI tests, full workspace typecheck/build, token gates, and map checks. The linked complete CI run passed before the latest focused fixes. - A disposable MinIO instance passed multipart upload, verified read-back, and cold restoration of an 18 MB archive. - Twenty warm samples with a synthetic 1,349-file, 22.8 MB repository retained five metadata queries and zero object reads. Median preparation was 256 ms for one selected skill and 691 ms for ten, with 5 ms emulated network delay each way. These are local measurements, not production latency guarantees. - No production instance was changed. Earlier verification failures and their disposition are retained below. <details> <summary>Earlier failures, review findings, fixes, and rerun evidence</summary> - The first CI pass found a stale inventory-span expectation and a missing repository-read span. Both are fixed; all three observability tests pass. One browser test copied page text into its YAML parser after reload; its first unchanged-head retry passed, but the race recurred after rebasing. The test now waits for the renamed URL and completed reload before copying the editor, retaining its original assertions. The earlier full local suite did not complete cleanly: it reported setup timeouts and assertions, while isolated relevant reruns passed. The complete current-head CI run closes that broad verification gap. No production instance was changed. - The first Apex review scored 2/5 and identified six issues. The fixes protect authored history even when it shares an in-flight Git upload, recheck upload expiry during orphan deletion, bound failed-upload cleanup, retain per-version kind/encoding metadata, add sanitized failure diagnostics, and correct the migration guide. The final focused run passes all 140 tests; the preceding eight-file run passed all 149 tests, including local storage and S3 diagnostics. Server typecheck and migration safety/numbering checks pass after the fixes. The second Apex review scored 1/5 and found unbounded maintenance storage calls and mismatched package/Git batch limits. Those are fixed with a 30-second cancellable cleanup deadline, shared 100 MiB/10,000-file scanner and storage limits, and bounded Git read batches. The proposed manual lockfile edit was rejected with evidence of the enforced CI-owned lockfile policy. All 19 storage/recovery unit tests and server typechecking pass. The new database-backed cleanup/conversion tests were locally skipped because the host exhausted its PostgreSQL semaphore capacity (confirmed directly by initdb). That gap is closed by current-head CI: [package-store tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109300889) pass all 18 cases and [package-worker tests](https://github.com/paperclipai/paperclip/actions/runs/40771320775/job/123109301157) pass all 7, with no skips in either suite. The corrected YAML browser shard passes. A separate planning-fixture test failed its cleanup DELETE with an activity-log foreign key violation after its behavior assertions passed; its one unchanged-head retry passed. The third Apex review scored 1/5 and identified a one-slot pool deadlock, projected path limits, cleanup holding the preparation lock during archive reads, and a publication/expiry race. These are fixed with transaction-local reads, prefix-aware bounds, isolated verification, and shared package row locks with a publication grace. The separate Paperclip review also found stale historical inspection, eager history downloads, and conversion retries blocking sleep. Those are fixed. Local PostgreSQL capacity recovered: the new database regressions ran, including the one-slot fork, paused storage during a warm turn, and a locked publication during expiry. The expanded run passed 144/145 tests; its remaining assertion still expected file bodies from the version list and was updated to fetch the selected version. The corrected company-skill suite passes all 81 tests, and the final package/projection run passes all 27. All 52 focused UI tests pass after updating their query-provider fixtures; the first UI run exposed a fixture remount, and the follow-up exposed an editing syntax error, both corrected. Full workspace typecheck and build, token gates, and map checks pass. The sandbox initially blocked the build tool’s local IPC socket; the authorized build rerun passed. Master then claimed migration 0401. The branch is rebased on f8226b0515, the unchanged SQL is regenerated as 0402, and repeated application passed twice on the combined schema. Post-rebase workspace typecheck passes. All 1,025 package, lifecycle, remote-launch, native-session, and asset-retention tests pass on the combined head. All CI checks passed for head 9d80a3b52d. The fourth Apex review scored 3/5 and found a save/edit lock-order cycle, a retry retaining an uncommitted source ID in its recovery recipe, and a summary read racing conversion. Commit b7d9237c6a fixes all three. The new concurrency and retry tests pass within all 61 package/source tests, and server typecheck passes. Master then claimed migration 0402. The branch is now rebased on 65fa1cd617 and the unchanged SQL is regenerated as 0403. All 68 package, source, and worker regressions, server typecheck, migration safety/numbering, and repeated migration application pass on current head 62553671a0. Subsequent verification and review outcomes are recorded below. - The fifth Apex review scored 4/5 and found cached version reads unnecessarily depending on object storage and conversion reads missing the coordinator cancellation signal. Commit 1eb90d80be fixes both: cached reads verify bytes without the preparation lock; manifest and archive reads carry cancellation through conversion. The first focused run passed 70/71 tests, including all three new cases, but the existing diagnostic test observed no warning. Its fixture now sets an explicitly expired migration deadline; the final worker suite passes all nine cases. Server typecheck passes. - The sixth Apex review scored 4/5 with one finding: concurrent cached inventories were unbounded. Commit 4b7589e14c gives cached inventories and previews a two-reader/32-waiter limit independent of cold storage. The first expanded pressure test observed a third reader before the preview path had the same bound; both paths now share the local limit. All 73 focused cases pass, followed by all 29 package cases after the final overload adjustment; server typecheck passes. The separate Paperclip review found no must-fix or should-fix issues. Its optional suggestion to persist durability verification was considered: cleanup intentionally verifies current external bytes before discarding the last local authored copy, and the missing-object regression requires that check. Shared-package eviction happens on cold fills with at most two attempts and a ten-second budget. - The seventh Apex review reached 5/5 with one remaining P2: timeout-bearing manifest reads bypassed request coalescing. The security review also found repeated archive acquisition after a cold administrative read. Commit 79a80a39d2 shares manifest loads with per-caller cancellation and cancels the underlying work when the last caller leaves; coalesces identical cold version/file requests; and publishes verified restored packages into the shared cache. Publication uses a short nonblocking lock, while eviction verification runs outside it. The first expanded run passed 73/74 tests and exposed a missing originating DB for cache publication inside read-only inspection; transaction readers now pass that owner explicitly, keeping metadata on the transaction. The corrected run passed 75/75. A final eviction test initially expected the shared package to remain after only its separate runtime projection was reused; it now explicitly accesses the shared package during verification. All 32 final package tests pass, including independent caller cancellation, one acquisition across concurrent/repeated reads, and warm preparation during stalled eviction verification. Server typecheck passes. - The eighth Apex review scored 4/5 with one finding: decoded cold inventories outlived the storage semaphore while waiting for cleanup. Commit 9d7921d1ec adds a separate two-response/32-waiter lifetime bound held through cleanup, preserving the independent warm-read queue and forwarding cancellation through eviction verification. The new regression pauses cleanup after two responses decode, verifies a third archive stays queued, and completes warm preparation before releasing cleanup. All 77 package/source/worker cases and server typecheck pass. On the preceding head, serialized server shard 6/9 was interrupted with “The operation was canceled” after 57 seconds of tests; the log contains no failing assertion. Fresh latest-head CI reruns that shard along with the other gates. - The ninth Apex review scored 4/5 with one P1: cold responses could own both permits while waiting on a one-slot ordinary database pool held by a queued transaction reader. Commit a94f0e409f reads caller-scoped metadata before admission and isolates all acquisition/cleanup SQL on a private connection. The new real-PostgreSQL regression reproduces the deadlock against the previous head and passes with the fix; all 78 focused cases and server typecheck pass. The previous CI run completed with 54 successful checks, including both formerly interrupted/stuck serialized shards. - The tenth Apex review scored 4/5 with one P1: preloading legacy bodies before admission could retain many full repositories in queued requests. Commit 276e353a37 removes that preload and also bounds legacy manifest derivation/fallback, using private connections inside the I/O slot. All 80 focused tests pass; the final synchronized package suite passes all 36 tests and server typecheck passes. - Apex review 11 returned 5/5 with zero comments on 276e353a37. While it ran, master 8bc051245f added migration 0403, creating a metadata conflict. Merge commit e1783764d9 integrates master and generates 0404_clumsy_leo.sql; its skill SQL is byte-for-byte identical to the previously verified migration. The combined schema passes migration safety/numbering, repeated SQL application twice, all 80 focused PostgreSQL tests, full workspace typecheck, and full build. The preceding a94f0e409f CI run was superseded and cancelled while its final runner job was active; no old-head success is claimed for that run. - Final acceptance: Apex review 12 returned 5/5 with zero comments on e1783764d9. All 28 review threads are resolved. The final head has 55 successful checks, two neutral reports, two skipped checks, and no pending or failing checks; the PR workflow completed successfully. Superseded run 40789121902 was cancelled to free its workflow slot for the final head; normal cancellation did not finish promptly, so force cancellation completed it. - October 29 conflict resolution (867c00673b): master 929e840595 added installed-selection reuse and bundled relocation, routine cleanup notifications, a hardened YAML copy helper, and migrations through 0413. The merge retains those behaviors together with package read scoping and the public-selection fast path; restricted selections still reauthorize and the full row read rechecks scope. Generated migration 0414 retains the existing SQL unchanged. Workspace typecheck/build and migration checks pass. Local database verification is unavailable: initdb reports semget / “could not create semaphores: No space left on device”; all 168 cases skipped, and repeated migration application failed at startup. Docker returned HTTP 500 during container creation, so no Docker test pass is claimed. Current-head CI must close this gap. - Apex review 14 scored 3/5 with one P1: a local import could copy cached private package files into a company-wide skill. Commit a2e597d800 rejects cache trees and ancestors through a canonical filesystem boundary, including aliases and ancestors checked before a concurrent fill. Ten local filesystem tests and server typecheck pass. The existing private-project integration case now rejects package-cache imports for both authorized and unauthorized users. The preceding merge commit’s complete CI run passed, and logs confirm all 168 locally skipped cases executed successfully: package store 36, sources 35, package worker 9, runtime selection 4, observability 3, authoring 81. - Apex review 15 returned 5/5 with zero new findings on a2e597d800. The separate Paperclip review found no must-fix or should-fix issues. Its optional suggestions were considered: unexpected filesystem cleanup failures remain visible, the durable marker cannot replace checking that bytes still exist before deleting a last local authored copy, and administrative history batching remains outside the pre-turn optimization. The cache-path literal centralization suggestion is also optional; the current producer and guard use the same directory, covered by the new integration regression. All review conversations are resolved. - Earlier October 29 CI/Apex acceptance: a2e597d800 was mergeable at that check, Apex review 15 passed 5/5 with zero findings, and all review conversations were resolved. The hosted verification above records the later acceptance limits and renewed conflicts. Workflow 40884087629 completed successfully, with 55 successful checks, two neutral reports, two skips, and no pending or failed checks. The project-skill-sources job log confirms all 29 cases ran and passed, closing the latest integration gap. The preceding merge workflow also confirmed all 168 focused database cases ran and passed. No local PostgreSQL or Docker pass is claimed. </details> ## Risks - First use after eviction can require storage or Git access. Revoked grants, removed sources, and unreachable commits can prevent reconstruction. Recovery never substitutes a newer commit. - Historical packages without a proven recipe keep durable bytes. Cleanup favors retention when a run pin cannot be interpreted safely. - New uploads and conversion use bounded work and persisted retries. Large initial conversions still add temporary storage and database load. - Optional cleanup can wait until the next server wake. Supported notifications assume one owning server process. - Per-entrypoint native tree duplication and cold preparation cost remain. The final ten-skill cold sample took 47.6 seconds on the shared host. - Generic authored edits can still hold an existing edit transaction while a package uploads. - Explicit file/folder deletion uses temporary disk space for rollback. An uncertain commit with a changed skill or unavailable database retains that backup and logs its location for recovery. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools, code execution, and tests. The exact served model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 12:19:37 -05:00
scotttongandPaperclip	71f0c0c99e	feat: add opt-in Coordinator workspace for chat, tasks, and decisions (#17983) ... <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/147/changes/paperclip/147 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - People coordinate work through conversations, tasks, and requests for input. > - These surfaces must share task state and preserve the context of each decision. > - The existing workspace makes people switch surfaces to follow that context. > - This pull request adds an optional Coordinator workspace with linked conversations, tasks, documents, and decisions. > - Each person can enable it in Settings > Experimental and return to the classic workspace. > - The benefit is a coherent path from a request to an assigned task and reviewed work. ## Linked Issues or Issue Description **Subsystem affected** UI, server, shared contracts, and database. **Problem or motivation** People need a consistent way to start tasks from agent conversations, answer questions, review work, and return to the source context. Decisions and task state must agree across these surfaces. **Proposed solution** Add a personal, default-off workspace experiment. Keep one conversation per agent. Add task proposals, compact task and decision cards, contextual document panels, and a Needs You queue with search, filters, sorting, grouping, and history. **Alternatives considered** Replacing the classic workspace for everyone would remove the opt-out path. Keeping separate decision state in each surface would create conflicting answers. This change uses the existing canonical task and interaction records instead. **Roadmap alignment** This supports the existing agent chat, task management, and human review capabilities. It is an experimental presentation choice. The user requested this isolated implementation after design review. No Discord discussion is claimed. ## What Changed - Add per-person workspace preferences, a default-off setting, and an additive migration. - Add Coordinator navigation, pinned destinations, task lists, agent switching, and contextual panels. - Link task suggestions and creation to the source conversation. Default new task ownership to the current agent unless the user specifies another owner. - Share pending questions, decision receipts, exact review targets, and history across tasks, chat, and Needs You. - Preserve task access, assignment authorization, revision binding, and required output review gates. - Add compact document links, relative timestamps, composer controls, and reduced-motion support for task and decision transitions. - Add component stories, regression coverage, and capability-map documentation. ## Verification - Final head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, based on master `8bc051245f77a47bee0f8ba45896b8bfa5922e0a`. Verified October 29, 2026 at 00:10 UTC: **MERGEABLE, 56 checks passed, four conditional skips, no pending or failed checks**. [Current CI run](https://github.com/paperclipai/paperclip/actions/runs/40790639688) passed without a rerun on this head. - Fresh Greptile **5/5**, zero new comments and zero open review threads; Superagent passed with no suspicious changes. [Checks](https://github.com/paperclipai/paperclip/pull/17983/checks). - Local workspace typecheck/build passed after the latest rebase. The final account-read patch passed UI typecheck/build and all 12 related preference/settings tests, including a delayed real-client request across account changes. Earlier scoped checks cover review deduplication/replay, decision deep links, cross-surface state, access boundaries, and migration replay. - Local database/browser rechecks after the latest rebase were limited by exhausted macOS shared-memory IDs: `initdb` failed before database creation. The local run passed 31 UI/snapshot tests and skipped 24 database tests; those skips and the blocked local browser start are not claimed as passes. The independent Linux CI database and browser checks passed. No running dev database or host setting was changed. - Upstream migration 0403 is preserved. Preferences and durable review retry aliases are generated as replay-safe migrations `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`; five newest snapshots are retained. Every detected conflict was rebased and pushed before waiting for CI/review, as requested. - The PR is unmerged. The active master ruleset still requires code-owner approval for the changed Paperclip skill; no approval or rule change was performed. <details> <summary>Earlier verification failures, integration history, and evidence</summary> These entries preserve prior checkpoints and failures; the current acceptance result is above. - Focused UI and server checks from the isolated build are recorded in the implementation plan. - Browser checks covered the opt-in shell, navigation, task/document links, and decision card states. No new live provider trial is claimed for this handoff. - Current acceptance head: `4b839aae1e314bd7e8635a766f1d205ff1c03ba6`, rebased onto master `8bc051245` immediately after its GitHub review-repair migration took 0403. Preferences are generated as idempotent `0404_tense_silver_sable.sql`, preserving upstream SQL/journal/snapshot history and five newest snapshots. GitHub reports MERGEABLE after the immediate lease-protected push. Scott's updated instruction is to rebase and push every conflict resolution first, without waiting for CI or Greptile; verification follows the push. - The final reviewed feature tree was consolidated for repeatable single-commit rebases. The complete previous commit history remains reachable through local recovery ref `codex/coordinator-recovery-before-rebase-20261028` (resolved snapshot `0e1bbdf4e`). The actual rebase completed, and its tree hash exactly matches that resolved snapshot. The force-with-lease required the known remote head `0e6ef7c44`, protecting against concurrent branch changes. No PR merge or permission/runner setting change occurred. Fresh CI/review are in progress for the current head. - `2001132fa` received Greptile 5/5 with one nonblocking account-switch read finding. Current head detaches path-coalesced preference GETs and disables HTTP caching, retaining React Query's account-scoped sharing. A delayed real-client GET regression proves account B can load its saved choice before account A responds, and A's late response only updates A's cache. Preference/settings tests 12/12 and UI typecheck passed; its review thread is resolved. Fresh current-head CI/reviews are pending. The prior CI chat shard 3 ended with “The operation was canceled” and no assertion failure ([job 123171595631](https://github.com/paperclipai/paperclip/actions/runs/40789688333/job/123171595631), `/tmp/pr17983-2001132-chat3.log`); no code/timeout change is attributed to that cancellation. - `843e823b4` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass. All browser tests including the repaired YAML shard passed, and 54 checks passed. General server shard 9 ended with “The operation was canceled” and no failed assertion; its test step was marked cancelled (job [123164350893](https://github.com/paperclipai/paperclip/actions/runs/40787501761/job/123164350893), log `/tmp/pr17983-843e-server9.log`). A single unchanged-head rerun was started after the workflow completed, but master moved while it ran. A local nested-clone check timed out in its 30-second database-startup hook and then failed cleanup before the 20 database cases ran; this is retained as a local verification limit (`/tmp/pr17983-843e-nested-clone-check.log`), not claimed as a pass or proof of the CI cancellation cause. - Following Scott's instruction, the new conflict was rebased and pushed immediately without waiting for that rerun. Upstream's `0403_natural_cerise.sql` is preserved; regenerated preferences and retry-alias migrations are `0404_tense_silver_sable.sql` and `0405_harsh_avengers.sql`, replay-safe with five newest snapshots retained. The YAML helper conflict combines upstream's click and header behavior with the editor-selection guard and injected regression. Recovery ref `codex/coordinator-recovery-843e823b4` preserves pre-rebase work. Current head is MERGEABLE. Full workspace typecheck/build passed; UI and snapshot checks passed 31 tests, but 24 database tests were skipped by the local support probe, and the local browser server could not initialize PostgreSQL. A direct disposable `initdb` diagnostic established the host limitation: `could not create shared memory segment: No space left on device` (`shmget`, shared-memory IDs exhausted, not disk space). No dev database or host setting was changed. Diagnostics: `/tmp/pr17983-2001132-initdb-probe.log`, `/tmp/pr17983-2001132-db-probe.log`; checks: `/tmp/pr17983-2001132-tests.log`, `/tmp/pr17983-2001132-browser.log`, `/tmp/pr17983-2001132-types.log`, `/tmp/pr17983-2001132-build.log`. These local skipped/blocked checks are not claimed as passes; fresh CI on independent runners is in progress. - `d18f683cd` reached fresh Greptile 5/5 with zero comments/open threads and Superagent pass, but [E2E shard 7](https://github.com/paperclipai/paperclip/actions/runs/40785966243/job/123159719925) failed parsing an Advanced YAML clipboard read beginning with “Skip to Main Content.” The log is retained at `/tmp/pr17983-d18f-e2e7.log`. Current head hardens the test helper to verify editor focus and selection, clear stale clipboard content, and require the YAML header at the start of a fresh copy before parsing. Saved-state assertions and timeouts are unchanged. All six Advanced YAML browser journeys passed on a disposable local instance (41.0s); the exact lost-response journey also passed with a one-time injected surrounding-page copy, verifying rejection/reacquisition (17.6s). The precise browser focus/clipboard timing in CI remains unproven; the observed invalid-read path is now guarded and deterministically exercised. Logs: `/tmp/pr17983-yaml-clipboard-scope-browser.log`, `/tmp/pr17983-yaml-clipboard-injected-browser.log`. Fresh current-head CI and reviews are pending. - `19e6dcb0b` passed Superagent and Greptile 5/5, with one nonblocking deep-link finding. Current head preserves attention links using `?decisionId=`: the requested pending decision is rendered even beyond the first 40 cards, focused and scrolled into view, then only that parameter is consumed so later refreshes do not refocus. Other URL parameters remain. Needs You tests 10/10 and UI typecheck passed; its review thread is resolved. The preceding post-rebase regression run passed 45/45 tests. Fresh current-head CI/reviews are pending. - `6d772b7cc` reached Greptile 5/5 with zero comments/open threads and Superagent pass. While 32 checks had passed, master moved and caused a documentation conflict in `maps/feature-map/onboarding.md`. Rebased immediately, preserving both onboarding paragraphs. The complete four-commit branch was pushed as `19e6dcb0b` before waiting on CI/review, per Scott's instruction. A transient Git index lock interrupted the first continue; the command sequence briefly pushed intermediate `34e298c07`, then the completed rebase was immediately pushed. All feature and security/retry fixes are present; local recovery ref `codex/coordinator-recovery-6d772b7cc` preserves the pre-rebase history. The only tree differences from that ref are upstream's two smoke-test files and the combined documentation paragraph. GitHub reports MERGEABLE. Fresh current-head verification is in progress; prior-head review results are not current acceptance. - `21efa3120` passed Superagent but Greptile scored 4/5 for a valid retry-key persistence bug: an acknowledged duplicate key could reopen a resolved review. Current head adds an additive, replay-safe `0404_light_swarm.sql` migration for indexed, task-scoped aliases from acknowledged retry keys to the canonical interaction. Alias registration and retry lookup occur under the existing task lock; payload/actor mismatch still conflicts, and each canonical review permits at most 100 aliases. Original and acknowledged keys replay the terminal result rather than creating a new gate. Five newest migration snapshots are retained. Service/routes tests 165/165 passed; final quality review plus migration replay/snapshot checks 25/25 passed; server typecheck passed. The P1 thread is resolved in code; fresh current-head reviews and CI are pending. - `69c2c7543` passed 55 checks and Greptile 5/5 with no open Greptile findings, but Superagent check [123140675812](https://github.com/paperclipai/paperclip/pull/17983/checks?check_run_id=123140675812) correctly flagged unbounded pending quality-card creation. Current head fixes that under the existing task-row lock: equivalent requests with fresh idempotency keys reuse the pending card; conflicting requests for an unresolved deliverable return the existing interaction ID; pending cards are capped at ten per task across all reviewers. Distinct outputs remain independent, and already-answered items in a partially pending batch can receive a fresh review. Added real-database concurrency, cross-card duplicate, partial-answer, and hard-cap regressions. Quality review 22/22 tests and final server typecheck passed; earlier service/routes 163/163 tests passed before the final partial-answer refinement. One added assertion initially expected a service HTTP error, but duplicate keys are rejected earlier by the existing shared schema; the assertion now verifies that validation. Fresh current-head CI/reviews are pending; the action-required security result is not treated as passed. - `c9c575e5f` completed all 56 checks successfully with four conditional skips, Greptile 5/5, and Superagent pass. Its two P2 review findings are fixed in the current head: independently bound quality approvals now each remain visible in attention; changing/clearing URL-backed assignee/status filters updates the URL and survives task navigation. Both review threads are resolved. Regression evidence: attention service 34/34 tests passed, CoordinatorTasks 20/20 passed, UI and server typechecks and the full workspace build passed on `69c2c7543`. The first new UI test run caught an assertion racing React Router's transition; waiting for the navigation before asserting passes without weakening the navigation/remount checks. These older-head green checks are not claimed for the new fix commit. - Preceding `0e6ef7c44` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, and passed Superagent. A fifth migration-metadata collision arrived before the final Runner job finished. Those green results are preserved as prior-head evidence, not current acceptance. - Previous fully verified migration head: `0e6ef7c44696a377f35e69521b93587576374bd0`, integrating master `f8226b051` after the fourth migration-number collision. Master added terminal GitHub-response migration 0401 while the final native check ran; upstream 0401 SQL/journal/snapshot is preserved unchanged, and preferences are generated as idempotent `0402_yielding_blink.sql` on top. Exactly the five newest snapshots remain. GitHub reports MERGEABLE after the push. All 1,155 relevant migration/review/interaction/native-execution/remote-assets tests, workspace typecheck/build, migration guards and whitespace checks pass. Fresh exact-head CI/Greptile/Superagent are pending. - Immediately preceding head `36c67cf03` completed all 56 checks successfully with four conditional skips, fresh Greptile 5/5 and no open findings, plus passed Superagent security. No retry was needed for that head's CI. Master advanced and introduced only a migration-metadata conflict while its final native check ran, so that green result does not establish acceptance of the current integration. Four consecutive collisions (0398, 0399, 0400, 0401) during verification are retained as concrete integration churn; a brief pause of competing migration merges was requested, without assuming it happened or changing any repo setting. The PR has not been merged. - Prior fully verified head: `36c67cf03`, on the clean `30059c0ed` integration of master `efd65c112`. The last Greptile P2 is fixed: Needs attention retains the participant-agent scope through authoritative server membership queries for each pending task, including historical participation when another agent currently owns the task. It does not approximate participation with current assignment or enumerate every company task. The actual view-switch regression and all 31 focused task/routing/collection tests, UI typecheck/build, token gates and whitespace checks pass. Greptile completed a fresh 5/5 review on this exact head (review 19) with zero new comments and no open review threads. CI and Superagent remain in progress; no current-head CI failure has been reported. - Head `30059c0ed` received Greptile 5/5 with that one nonblocking participant finding, now fixed. Superagent passed that head. Current-base integration restored the ephemeral RunsOn fleet after a superseded workflow retained queued aggregate jobs: normal cancellation was insufficient; the single force-cancel request was interrupted but readback confirmed that obsolete run cancelled and current CI began on the trusted fleet. No cancellation retry or runner/security/workflow changes were made. Its partial CI was superseded by the necessary participant fix and is not current acceptance. - Previous compatibility/base-sync head: `30059c0ededc0eab4e04e8299a884f36db8b292f`, integrating master `efd65c112` and the Inbox/workspace compatibility fixes. GitHub reports MERGEABLE after the push. Coordinator keeps Inbox standalone under More without rewriting the instance's Combined Inbox preference; opting out restores that preference. Saved merged-Inbox URLs resolve to their original owner, task status presets remain usable, and canonical repeated `workspace` parameters retain scoped, deduplicated pagination (including Needs attention). All 192 broader routing/Inbox/sidebar/privacy/review tests and the final 16 task-page tests pass, as do workspace typecheck/build, token gates and whitespace checks. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The preceding `d5b9be5bd` remained mergeable but its CI routing gate intentionally fell back to GitHub-hosted runners: “stacked base merge commit does not contain the current base branch.” All 45 downstream jobs stayed queued. Current master was merged cleanly to refresh the pinned base; no runner/security settings or workflow logic were changed. Greptile scored that preceding head 4/5 with the two compatibility findings fixed here. Its automatic Superagent scan was inconclusive at checkout; the documented retry was still pending when superseded. Those results do not establish acceptance of this current head. - Previous approval/migration head: `d5b9be5bd3b8d6b0437e42bf48138377f34fa67d`, integrating master `ba5bba86d` and the stale-output approval correction. GitHub reports MERGEABLE after the push. The fresh P1 finding is fixed: every submitted item now validates and locks its bound output inside the resolution transaction, whether or not it carries a required-quality-review binding. Real database regressions verify that an advanced document revision or deleted attachment refuses the entire batch without recording approvals, and current ordinary output reviews still resolve. All 272 combined migration/review/interaction tests pass; workspace typecheck/build passed. Fresh exact-head Greptile and CI are pending. The automatic Superagent scan ended inconclusive at checkout; one documented on-demand retry is pending. These are explicit verification limits, not passed security results. The P1 thread is resolved. - Master consumed the next migration slot three times during this verification pass: confirmation recovery took 0398, continuation recovery took 0399, and GitHub acknowledgement recovery took 0400. Each upstream SQL/journal/snapshot is preserved unchanged. Preferences are now generated as idempotent `0401_white_lenny_balinger.sql` from upstream 0400; exactly five snapshots remain, and replay coverage locates this migration by table content. This is recorded concurrent integration churn, not a claim that older green results apply to the latest base. - Head `ed3b68da9` fixed the two history/URL findings with 26 focused regression tests, server/UI typechecks, workspace build and token gates passing. Greptile then scored that head 4/5 with the stale-output P1 corrected in the current head. Its partial CI is superseded, not counted as current acceptance. - Previous filter-fix head: `ed3b68da9`, based on the `8458d39d9` merge of master `063cfd438`. The two fresh Greptile findings are fixed: history outcome filters now match recorded per-item verdicts in SQL before pagination (mixed reviews remain in both applicable outcomes), and same-page URL assignee changes/back/forward navigation update that filter without clearing independent filters. All 26 focused database/navigation/collection tests, server/UI typechecks, workspace build, token gates and whitespace checks pass. The two review threads are resolved. Fresh exact-head CI, Greptile and Superagent results are pending. - The `8458d39d9` migration integration also passed all 328 targeted migration/interaction/confirmation tests, workspace typecheck/build and Superagent security scan. Greptile scored that head 4/5 with the two functional gaps fixed in the current head; those prior-head checks are not current acceptance. - Migration integration head: `8458d39d9721f4cb312c2390070a0609f8cddd1a`, integrating master `063cfd43813cabf18417605fb9eefb9aacf95c8d`. GitHub reports MERGEABLE after the push. Master advanced twice during this conflict-resolution pass: first its confirmation-recovery migration took 0398, then its continuation-recovery migration took 0399. Both upstream migrations and journal entries remain unchanged; Coordinator preferences are now generated as idempotent `0400_sad_guardian.sql` from upstream's latest snapshot. Exactly the five newest snapshots remain. The replay regression now locates the table migration by content instead of hardcoding a renumbered filename. - All 328 targeted question/review/confirmation/migration tests passed again after the second merge. Workspace typecheck/build and fresh exact-head CI/reviews are in progress. The preceding `0c46f46d6` passed workspace typecheck/build and Superagent security scan, but was superseded because master consumed migration 0399 while those checks ran. That prior-head result is not acceptance of the current head. No production settings or runner/security permissions were changed, and the PR has not been merged. - Previous integration head: `0c46f46d6f2b23417ab872ce3cb6ff2673647287`, integrating master `287b04d72f1b9fca1412e05435c75b8302ba0710`. GitHub reports MERGEABLE after the push. The renewed conflicts were `0398_snapshot.json`, `_journal.json`, and the top imports in `issue-thread-interactions.ts`. Upstream's `0398` confirmation-recovery index and journal history are preserved unchanged; Coordinator preferences were regenerated as idempotent `0399_bouncy_miek.sql`, with replay coverage updated and exactly the five newest snapshots retained. Both terminal confirmation notifications and Coordinator task assignment/required review behavior are preserved. - All 328 targeted interaction/review/confirmation/migration tests pass, as do workspace typecheck, map validation and whitespace checks. The workspace build and fresh exact-head CI/Greptile/Superagent checks are pending. The first migration-generation attempt hit the sandbox's local IPC restriction before generating files; the unchanged authorized local generation completed successfully. Prior-head green checks below do not establish acceptance of this merge. - Previous acceptance head: `8b7dcf739ed15afc51351fdfa0e0e5fbc75ab788`, with master `1aa3a21ac` integrated and GitHub reporting MERGEABLE. Greptile completed a fresh 5/5 review on this exact head with zero new actionable findings and no open review threads. The [fresh CI workflow](https://github.com/paperclipai/paperclip/actions/runs/40752864319) completed SUCCESS on attempt 2 at 19:40:08 UTC, including all eight E2E shards. Final completed check set: 55 succeeded, four conditional checks skipped, zero pending/queued/running checks, and one neutral/inconclusive Superagent scan. The auxiliary commitperclip review and isolated native Runner compilation/cache-reuse check both passed. This is not a claim that Superagent produced a passed security verdict. GitHub still reports MERGEABLE against current master `60b8089a44b4eaa37e627adbfdf2f130bda13335`. - Current-head E2E: all eight shards and the aggregate passed, including the previously failing YAML browser journey. Serialized server shard 9/9 failed: the protected YAML projection test exceeded its original 15-second timeout; its late update request then reached the next test's reset mock (the captured payload is the prior test's model/label update). All 88 tests in that exact unchanged file passed locally in 38.03 seconds, with no timeout or assertion changes. The original slow-request cause remains unproven. CI failure log: `/tmp/pr17983-8b7-serialized9.log`; local recovery: `/tmp/pr17983-adapter-yaml-repro.log`. The single bounded unchanged shard rerun passed, and the workflow completed SUCCESS. Rerun log: `/tmp/pr17983-8b7-serialized9-rerun.log`. The initial failure is retained; a passing retry does not establish its original slow-request cause. - Greptile scored preceding head `4873704be` 5/5 with one valid P2: opened task decision records missed company-scoped live invalidation. The current head uses the existing company decision prefix for resolution and effect-completion events, and resets private record payloads on access changes. Its thread is resolved. All 107 focused record/live-update tests, UI typecheck/build, token checks and whitespace checks pass. - CI on `4873704be` failed E2E shard 7/8 in the Advanced YAML lost-creation-response journey: its clipboard helper selected page text starting with “Skip to Main Content” instead of YAML. The exact unchanged journey passed in a disposable local browser instance (1 test, 17.1 seconds). No assertion, timeout or production behavior was weakened; the original failure cause remains unproven. Failure log: `/tmp/pr17983-487-e2e7.log`; local check: `/tmp/pr17983-agent-yaml-repro.log`. The full unchanged Advanced YAML browser suite also passed locally (6 tests, 37.8 seconds). Fresh current-head CI subsequently passed this journey and all other E2E shards. - Both the automatic and one supported on-demand Superagent scan on `4873704be` ended neutral/inconclusive: “Pinned PR scan merge-base checkout failed or timed out.” These do not count as a passed security verdict. The current head's automatic scan and one documented on-demand retry also returned the identical neutral checkout failure, with zero annotations or further diagnostics. This is an external scanner blocker, not a passed security verdict. No security settings or permissions were changed. - Recorded decision links now render and focus the full source-task record in both Classic and Coordinator, including recorded choice, submitted inputs, and the existing effect-result resolver. Fresh terminal records take precedence over older open-decision cache entries. Context and account checks keep mismatched or inaccessible records out of the DOM; no decision-history tab was added to tasks or chat. The shared company live-run feed now drives live-only filtering, including ended runs and terminal tasks with stale run entries. All 213 focused tests, UI typecheck/build, token and whitespace checks pass; the final focus follow-up passed all 188 task-detail tests and UI typecheck. The first new navigation test failed because its test host omitted company context; the host was corrected before the passing run. - Previous integration head: `375c5e89516705050d94bab99336212841b2e6e1` merges master `1aa3a21ac` after four additional upstream commits. The sole conflict was competing imports in `server/src/services/issues.ts`; both quality-review policy and inbound wake-notice notifications are preserved. The PR-specific review import now sits beside execution-policy imports instead of the frequently changing first line. GitHub mergeability is rechecked after the push. - On the combined master tree, workspace typecheck/build, all 153 affected UI tests, map validation, and whitespace checks pass. Four server suites passed 77 tests; the fifth suite's module-import hook timed out while broad verification was running. Its unchanged isolated run passed all 41 tests in 7.2 seconds with the original timeout. This records recovery without claiming the original import-timeout cause is proven. CI on `375c5e895` passed every substantive job, including all eight E2E shards and Superagent; Greptile found the two functional gaps fixed in the current head. A short-lived `669d90876` commit added the main fixes before the focused navigation follow-up. - Immediately preceding head `9d9737b49` completed all 56 checks successfully with four conditional skips, Greptile 5/5 and no open findings, plus a successful Superagent scan. The signoff E2E shard passed on its single accepted rerun; its initial cancellation/timing failure is retained below. Master moved again while those checks ran, requiring this final integration rather than treating the prior head's green result as current acceptance. - Previous head `9d9737b496911a6054d0ea22d1b42b28b115e9ab` includes master `d30c8f42b842327ee1c18c41a2efd6ba623da5cd`; that head was subsequently conflicted by moving master. The master merge preserved durable account-bound writes and Coordinator UX. Workspace typecheck/build, 416 focused UI tests, and 142 affected comment-route tests passed. A stable full UI run on preceding head `938ad1665` passed all 9,833 tests across 836 files; its final full workspace build also passed. - Greptile found nested proposal project inheritance and Classic bound-revision links on `91be74952`. Both were fixed with 142 interaction-route tests, 113 focused UI tests, and UI/server typechecks. Its next review identified a valid draft-preservation bug in the new Classic loading UI. Head `9d9737b49` keeps loading/error notices outside editors and annotations while preserving dirty drafts; 115 focused review UI tests pass, including pending/failed revision loads and continued draft editing. All related threads are resolved. Greptile completed a fresh 5/5 review of `9d9737b496911a6054d0ea22d1b42b28b115e9ab` with no open findings. - CI on `938ad1665` exposed an integration regression with the newly merged durable startup saves: the personal preference gate blocked the task page before its first session lookup. The guarded title editor now mounts while that lookup is pending, with presentation-specific tools still gated. All 216 affected component tests pass. The unchanged five-test `disconnect-write-recovery.spec.ts` browser suite passes against a newly launched disposable local-trusted PostgreSQL instance on port 33219, including the exact failed startup-save journey. No live provider, hosted authentication or production proof is claimed. The helper-owned test server exited and the port has no listener. UI typecheck, built-browser assets, token and whitespace checks pass. - Initial durable-write merge tests found duplicate imports and unscoped draft fixtures, fixed before 416 focused tests passed. A sandbox typecheck could not create its IPC socket; the supported rerun passed. The first revision-link test used a wrong Return to latest button assertion, corrected before the 113-test pass. The startup E2E reproduction initially still failed because an outer shell gate also blocked the page; the subsequent fix passed all five cases without changing the test or its timeouts. A focused loading-header assertion initially used an incorrect fixture title; correcting it preceded the 216-test pass. These failures remain recorded. - Earlier routing gates fell back to GitHub-hosted runners while master was moving or conflicted. Superseded queued runs were cancelled, then the `938ad1665` routing gate explicitly selected the ephemeral RunsOn fleet. Its CI actively executed and completed most tests; only the diagnosed startup E2E shard and aggregate failed, alongside the valid Greptile finding. Auxiliary GitHub-hosted workflows also completed successfully on earlier heads, so queueing alone is not an administrative blocker. Fresh [CI on `9d9737b49`](https://github.com/paperclipai/paperclip/actions/runs/40744012678) is the acceptance target; the previously failing startup E2E shard now passes. A different signoff E2E shard failed during executor resubmission when its fixture run was cancelled as `issue_continuation_waiting_on_review`. The exact unchanged journey passed locally (1 test, 46 seconds), and the preceding head with identical server/signoff code passed that CI shard. This supports a bounded rerun but does not establish the timing failure's root cause. After the workflow finished, GitHub accepted one rerun of that failed shard and its dependent aggregate; both passed, completing all checks on that head. The failure log is retained at `/tmp/pr17983-9d9737-e2e7.log` and the local reproduction at `/tmp/pr17983-signoff-repro.log`. No assertion or timeout was weakened. - On the first merge, full workspace typecheck/build, all 9,622 UI tests, 267 tests across seven affected server suites, two migration checks, map validation, token gates, and the PR diff whitespace check passed. Greptile then found duplicate local task-search filtering and lost completed item-review detail. Both are fixed, with 42 passing focused UI tests and passing UI typecheck/build. - A subsequent Greptile review found attachment confirmations without working links and mobile history without a visible host. Confirmation layouts now derive the authenticated immutable-content endpoint from the attachment ID, and Needs You uses the existing full-screen mobile sheet. All 119 focused tests pass, including missing/stale display URLs, both review layouts, and both mobile history entry buttons. UI typecheck/build and token checks pass. Full workspace typecheck/build also pass after the second master merge. - Greptile gave head 22f69c304 a 5/5 score with one non-blocking desktop-history reopening finding. Head 6dd52b6d3 fixed it, passed all nine history tests, UI typecheck/build, token gates, and the full 9,679-test UI suite, then received a fresh 5/5 review with zero open findings. These are prior-head results; review on the master-sync commit is pending. - The chat and workspace CI jobs on head 22f69c304 were interrupted by runner shutdown signals and cancellation before completing their tests. Their logs do not report failing assertions. Those interrupted jobs are not claimed as passes. The new head starts fresh runs. - The first merge CI had a Slack session-sync idle assertion fail after an unexpected database read. Its exact isolated reproduction passes, and the chat CI shard passed on head 618c5a2a3. The original failure's cause remains undetermined. The broader local reproduction, run concurrently with full verification, instead encountered 21 timeout failures across its two test chunks; it is not claimed as a passing full local server run. A concurrent local UI run had eight failures (seven timeouts and one delayed autosave assertion). The complete UI run on the final head passed all 9,679 tests across 830 files with two workers; no test timeout or assertion was weakened. This establishes recovery without claiming that every earlier timeout's cause has been proved. An earlier intermediate UI run also overlapped the second master merge and failed the newly added Advanced-sidebar expectation, so it is not treated as stable-tree evidence. The newly merged UI dependencies initially required refreshing the local install from the upstream package manifest; the generated local lockfile update is not committed, per repository policy. - Superagent returned neutral/inconclusive scans on the preceding heads because its pinned merge-base checkout failed or timed out. The available retry endpoint returned 404. These are not successful security verdicts. The fresh scan on `91be74952` passed (PR scan passed; no suspicious PR changes detected). The `938ad1665` scan again returned neutral/inconclusive with the same pinned merge-base checkout error; the check-suite rerun endpoint returned 404. The fresh `9d9737b49` scan completed successfully at 18:22 UTC with no suspicious changes detected. Earlier neutral scans are not counted as successful security verdicts. - Prior head `57902152ac3218096fca669d33ff18400a2a86ae` passed 9,618 UI tests across 826 files, received Greptile 5/5 with no open findings, and completed CI with 56 passes and four conditional skips. These are historical results, not acceptance of the latest merge commit. - The first full build found generated capability-contract drift. The generated outputs were refreshed. Initial sandbox runs could not open local IPC sockets; verification was rerun with local test access. - Current master was merged. The workspace preference migration was regenerated after the existing migration history. - The UI run first found two outdated expectations: early custom-answer validation and the document access error copy. Both were corrected, and the full UI suite then passed. - The local server run found three failures in unchanged tests. The Muse test passes with Python 3.13 instead of the host's Python 3.9. The two workspace tests pass when their test process uses the expected master default branch. These environment settings do not change application behavior or global machine settings. The unsharded server run was stopped after those reproductions; CI supplies the full sharded server result. No successful full local server run is claimed. - Skill inventory generation needs an external eval checkout, which is absent locally. The existing generator functions refreshed only the live skill inventory and its derived files. The checked-in eval inventory was preserved. The inventory validation gate passes. - Greptile found a dropped MCP review requirement, missing per-item output links, generic item-review summaries, and a history-refresh failure. The fixes have 32 passing database-backed server tests and focused rendered-card tests. A follow-up partial-review finding was corrected by forwarding the canonical mutation status before settling the card. Attachments open through the authenticated immutable-content route in both card layouts. The final focused UI set has 117 passing tests, including these regressions. - Both browser shards that first failed expected a disabled-chat message. The copy was restored, and those shards passed in CI on the next commit. The two journeys also passed locally. One local attempt first timed out before feature disable while waiting for the provider's initial message; its isolated reproduction passed. That earlier timing failure remains recorded and is not presented as an explained infrastructure failure. - Serialized server shard 9 first failed during pnpm setup with ECONNRESET, before dependencies or tests ran. Its single rerun passed setup and all tests. The build and production runtime boot passed on the first attempt, but that run's public-install step timed out during Pi setup. The single build-job rerun passed compilation, runtime boot, and clean public-install verification. Those setup scripts are unchanged by this PR. The successful retry establishes recovery; the earlier Pi timeout's underlying cause remains undetermined. The aggregate CI gate is green: [workflow run](https://github.com/paperclipai/paperclip/actions/runs/40637789695). </details> ## Risks - This touches shared task, interaction, and completion paths. Regression tests must cover classic and experimental behavior, authorization, duplicate answers, and stale review targets. - The additive preference table must migrate before the setting is used. The setting defaults off. - The experiment changes navigation and message presentation. Canonical task and document records remain the source of truth. - The initial task brief and saved plan can repeat information. This change fixes chronology; a broader redundancy policy remains a design decision. ## Model Used OpenAI GPT-6 Codex with reasoning and tool-assisted code editing, tests, and browser verification. Earlier iterations used Codex models in this task; their exact model IDs are not available in the current context. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used - [x] I have checked ROADMAP.md - [x] I have searched GitHub for duplicate or related PRs - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change - [x] I have run relevant tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open recommendations or follow-ups - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 17:20:01 -07:00
DottaandPaperclip	b14e1ecaf8	feat: add a complete agent YAML configuration editor (#18036) ... <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/175/changes/paperclip/175 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Operators configure each agent through several settings tabs. > - Operators also need one complete, editable configuration document. > - The document must retain each resource service's access and validation rules. > - This pull request defines the `paperclip/v1` Agent YAML format and adds an Advanced editor. > - The editor previews changes, rejects stale core writes, and explains partial saves. > - Operators can edit agent settings in one place and verify the saved state after reload. ## Linked Issues or Issue Description Refs #2219. Related settings work: #10974. **Problem or motivation** Agent configuration spans profile, runtime, instructions, skills, tools, channels, permissions, secrets, keys, and revision history. Operators cannot inspect and edit those settings as one versioned YAML document. **Proposed solution** Add an Advanced agent tab with a YAML editor. Define a shared document schema. Read and write resources through their existing APIs. Separate editable settings, explicit commands, and read-only observed state. **Alternatives considered** A raw agent-row editor would omit related resources. A generic database update would bypass resource ownership. The editor therefore uses the existing resource contracts and records partial results when a later request fails. **Roadmap alignment** This extends the shipped agent configuration capability. It relates to easy AGENTS.md configuration, multi-harness teams, governed tool access, and secrets management. Stored credentials remain redacted. This format is an editable settings view; it does not add company portability or provider consent. ## What Changed - Add the `paperclip/v1` Agent document schema and format documentation. - Add the Advanced route, sidebar entry, YAML editor, validation, change summary, save progress, and reload recovery. - Cover core settings, instruction files, skill pins, tool installs/profiles/policies, channels, permissions, secret operations, API keys, and revision restore. - Add private-keyed configuration revisions checked under the agent write lock. Read the current key through its owner so file-based rotation does not require restarting the server. Preserve unchanged secret markers at their original paths. - Retain drafts and returned credentials after partial saves. Require reload before retrying commands. Prevent duplicate submissions. Keep new key tokens visible through rename and disable saves with an empty change plan. - Send only changed channel settings. Require explicit intent for shared profile edits. Load destination instruction files when changing their location. - Update Storybook fixtures, verification maps, and regression tests. Add browser acceptance through real APIs and PostgreSQL. ## Verification - `pnpm -r typecheck` passes. - `pnpm build` passes. - The focused UI suite passes all 50 tests. The API/revision helper suites pass all 93 tests. - All 65 lifecycle tests pass with a dedicated PostgreSQL database. They exercise real row locks, redaction, revision history, audit, and company access. - All six browser acceptance cases pass. They check saved settings and clears, secret redaction, key creation/revocation, stale drafts, lost-response recovery, instruction relocation, revision restore, and key-token retention through rename. The tests use real resource APIs and storage. The MCP target uses the existing local provider fixture. No model run or provider login is claimed. - Signing-key regressions cover both environment changes and real in-place key-file rotation. The protected database save/fencing cases pass after removing the cache. A local measurement of 1,000 file-backed revision calculations took 48 ms. - `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and Storybook catalog checks pass. - The full [PR CI run](https://github.com/paperclipai/paperclip/actions/runs/40725425470) passes on `34ed97bbf9`: all 48 jobs, including all eight browser shards. Greptile is 5/5 on this commit, with no unresolved review threads. This final commit refreshes CI after a runner-routing fallback; its source tree matches `fb4cf488cf`. - Paperclip Review and Superagent Security Scan ended neutral because their external agents did not complete a valid assessment/receipt. Superagent passed on `fb4cf488cf`, which has the identical source tree. One request to retry the final scan returned HTTP 404. Neither check reports a code finding. - The local full-suite attempt was blocked by host System V semaphore exhaustion. Acceptance and lifecycle verification used an isolated PostgreSQL 18 container. Earlier CI runners were interrupted. A local combined run hit hook/test timeouts; the full 88-test route file passed after the builds finished. The final full CI run passes. Earlier failure logs and browser traces remain retained outside the repository. To run the browser regression: ```sh pnpm exec playwright test --config tests/e2e/playwright.config.ts tests/e2e/agent-configuration.spec.ts ``` ## Risks - Multi-resource saves are sequential. Earlier operations can remain saved if a later operation fails. The editor reports the completed steps and requires reload. - Core settings and managed instruction files have atomic revision/hash checks. Other unversioned resource writes and external supporting files retain a race window after preflight. - Shared profile definitions affect other bindings. The editor requires `updateSharedDefinition: true` for those edits. - The editor follows native service normalization and experimental settings. Canonical reload can add generated access profiles or mask environment values. - New editor dependencies require the repository's normal automated lockfile update. This pull request does not edit the lockfile or add a database migration. ## Model Used OpenAI Codex, GPT-6 family. The exact current model version was not exposed. The work used reasoning, repository tools, code execution, and browser automation. Earlier work in this chat used the configured Codex model before a model switch; its exact model ID and context window were not exposed. The current context window size is also not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 11:49:24 -05:00
DottaandPaperclip	6cc765bd81	feat(live): deliver recoverable transcripts to interested views (#17456) ... ## Thinking Path > - Paperclip should show live agent output without repeating presentation work for every view. > - Execution, acknowledgements and durable checkpoints must remain independent of browser interest. > - The server already owns authorization and redaction at delivery. > - A shared browser connection can declare which visible runs need full transcripts. > - Catch-up must handle subscription races, disconnects, terminal output and slow consumers. > - This PR delivers the backend and UI contract together without a database migration. ## Linked Issues or Issue Description **What existing behavior does this improve?** Visible transcript consumers receive recoverable output through one shared browser connection. **Current behavior** Independent views can create duplicate connections, presentation work and recurring history reads. A subscription race or filtered native history can lose the saved tail. **Proposed behavior** Negotiate run interests, subscribe before catch-up and retain independent canonical recovery cursors. Keep lightweight lifecycle and attention events available without full transcript delivery. PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is independent of hot-upgrade recovery; the stack order is for review convenience. ## What Changed - Add additive WebSocket capability negotiation, replacement interests, revision acknowledgement and resynchronization notices. - Reference-count visible consumer interests on one company-scoped browser connection. Preserve older-client/server behavior. - Reuse canonical structural/current-user/registered-secret redaction for plugin run notifications through the same bounded delivery queue. Keep producer and uninterested streams free of recipient lookups. - Refresh application queries after the matching recovery acknowledgement, including one follow-up after an older in-flight read. Ordinary interest acknowledgements add no refresh. - Keep authorization and redaction at delivery. Browser interest cannot change harness execution, PRP ACKs, checkpoints, accounting or canonical logs. - Deduplicate by sequence and log identity. Catch up terminal output and saved tails without recurring reads during healthy negotiated streaming. - Bound delivery to 1,000 items and 1 MiB. Overflow requires resynchronization; persistent congestion closes the connection. - Preserve compact-history expansion from current master across paginated catch-up. Snapshot the original read window before React can defer its update. ## Verification On current source `f6710f0b3c`, Linux [job 122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503) executes all five run-read permission cases successfully (2.574 seconds), including grant and revocation on the same connection. Its shard passes all 2,430 cases. Current [server-shard job 122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416) has 1,513 passes, one failure and three skips: an unchanged nested-clone expiration/restore test exceeds its 15-second deadline. The preceding `5efc6c48bf` source has identical non-document bytes; [its corresponding job 122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272) passes all 21 nested-clone cases, including that case in 9.505 seconds. The exact timing cause is unproven. The single diagnostic [retry job 122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597) passes all 21 nested-clone cases (61.316 seconds), including the formerly timed-out case in 7.003 seconds, and all 1,514 shard cases (three skips). The failure remains recorded; no timeout or assertion was changed. After the server squash merge and qualification retarget, all 50 focused server cases and all 91 focused UI cases pass again, with both typechecks and token gates. Runner vendor preparation rebuilds current master inputs before those tests. This source needs fresh CI and reviews; prior CI and browser receipts do not qualify it by inference. Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after merged #17097, #17439 and #17455. Its diff contains only transcript delivery and its documentation. It adds no database migration. Retargeting changes only the evidence document; all production and test code is byte-identical to `5efc6c48bf`. On the eighth integration, all 50 focused server delivery cases and all 91 focused UI cases pass, with both typechecks and token gates. The following 35-case receipt belongs to the earlier delivery correction. Four low-queue authorization/decoration stall variants and the late-rejection case fail before correction. They now verify the five-second read deadline, old and negotiated clients, revisions that cannot reset the deadline, unaffected sibling connections, rejected late results, and timer cleanup. The first combined toast run has four loopback EPERM failures and 31 passes; its permitted repeat passes all 35. Earlier 28-case evidence excluded the toast file. The permission correction requires the current HTTP company-scope transcript permission, plus task access, for both native events and legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes all five run-read cases, including grant/revocation on the same connection, successfully. That source placed this file in server shard 11; later shard assignments can differ and must be checked against their actual inventories. The older `405bedd189` receipt retains a separate scheduled-retry teardown race and inherited fixture collision; neither is waived. The new head must pass again in Linux CI. The seven extracted stream-recovery tests preserve their assertions. All 39 related legacy/native cases, UI typecheck and token gates pass after restoring the initially missed fixture helper. Earlier full workspace build/typecheck and Chrome journeys at `445a76563c` cover two views, company switches, reconnect, terminal catch-up, reload and zero recurring healthy transcript reads. Those browser receipts are historical source evidence and do not qualify this new head by inference. The [qualification record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md) retains exact earlier UI/history, source-boundary and cleanup failures with their fixes and limits. The workload-cost test preserves bounds and authorization callback counts for zero interests, one viewer, multiple browsers and a slow viewer. Its socket congestion is simulated. It measures neither SQL nor network costs. Current permission checks and the final hosted active-agent/viewer matrix remain unmeasured; no production savings are claimed. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and ordinary merge eligibility remain required. The earlier recipient corrections pass all 47 registry/plugin/delivery/WebSocket/toast cases and all 174 UI provider/recovery/interest cases, both typechecks and token gates on source `b202e59a25`. A real bus/host-service test registers a synthetic run secret through the production registry and reproduces its raw notification before the fix; native/legacy/final notifications now redact it, disposal prevents late worker notification. A later P1 review reproduces lost final notifications on settings-read and secret-resolution failure. Both cases fail before correction. Any delivery failure or subscription timeout now sends one content-free stream error and removes the plugin subscription. It never changes the agent execution or run status. Later events start no new redaction reads. The final integrated six-file server lane passes all 50 cases, with server typecheck and token gates. Existing browser delivery behavior is unchanged. Storage and secret-provider seams are scripted. Two real React Query/provider regressions first miss a run during resync; both now refresh after the valid replacement ACK, including pending-read and stale-scope cases. The prerequisite qualification batch passes all 118 cases and E2E typecheck after master integration. Fresh CI, security scan and reviews remain pending on this head. The contribution-rule review correction removes 104 explanatory comments introduced by this slice and keeps the ordering, deadline and plugin-delivery explanations in the delivery document. Existing comments and functional test/lint directives remain. All edited source files have identical parsed non-comment tokens before and after the change. All 50 server delivery cases and all 91 cases in six affected UI suites pass, along with token gates. The new head requires fresh CI, security and review. ## Risks A slow or disconnected viewer must resynchronize from authorized canonical history. It may see a display gap until catch-up completes. Company changes and revoked access must remove delivery interests. This slice adds no database migration and cannot change execution or recovery authority. Full transcript delivery evaluates current company-scope permission per delivery. Active plugin recipients also perform canonical redaction reads; viewer-load database costs on this final source remain unmeasured. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 22:23:30 -05:00
DottaandPaperclip	bd382eee53	fix(recovery): explain retained source verification failures (#17773) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Workspace recovery must prove the identity of retained work before it uses an allocation. > - The verifier rejects several different contracts with the same `source_binding_invalid` code. > - A saved Retry can therefore stop before file preservation without enough evidence for a safe repair. > - Operators need a bounded observation of the selected source without another attempt or weaker guards. > - This pull request records closed failure reasons and adds an explicit read-only inspection on the run page. > - The benefit is a precise next diagnostic step while the saved work, intent, and ownership fences remain intact. ## Linked Issues or Issue Description Refs: #17741, #17053. This is a diagnostic follow-up. It does not change recovery admission or the retained-source acceptance policy. **What happened?** A retained-source recovery exhausted in `verify_execution` with only `source_binding_invalid`. Run and task diagnostics could not distinguish an invalid reference, lease policy, source binding, or reopened allocation. The normal recovery-actions read can also revalidate recovery state, so it was not a safe substitute for a current read-only inspection. **Expected behavior** Record an allowlisted reason at the rejected guard. Allow a reader to inspect one exact task/run/action without changing its attempts, saved input, receipts, or ownership. Clearly separate the recorded failure from current identity observations. **Steps to reproduce** 1. Prepare a terminal native run with a retained-source action and a lease policy of `reuse_by_environment`. 2. Attempt source verification. The guard must still reject that policy. 3. Open the failed run and choose Inspect recovery, then Inspect current source. The diagnostic should name `lease_policy_not_retained` and the allowlisted policy. ## What Changed - Split existing source-binding guards into closed reason codes. Keep each acceptance predicate and physical recovery action unchanged. - Persist validated, path-free reason/policy detail with future system recovery failures. Keep historical generic records compatible. - Add an explicit query on the existing task recovery-actions GET. Exact company/task/run/action selection bypasses read revalidation. Run selection lists at most 20 actions; inspection stops before per-lease lookup when a record has more than eight sources. - Add a manual inspection control on native export-failure run pages. Keep resolved retained actions inspectable. Show recorded failure and current recorded-source identity separately. - Update execution semantics and feature/access recipes. The shared type is the normalized server/UI contract; it contains no provider payload or secret. ## Verification - RED: the prior verifier failed both reason assertions for rejected warm policy and foreign-run lease binding. GREEN: the final focused source/HTTP, system-state, and UI suites pass 23 tests. The UI follow-up passes 8 tests after the review fixes. - HTTP tests prove exact selection, cross-company denial, resolved-action access, reused-allocation rejection, redaction, and unchanged actions, runs, leases, tasks, wakeup receipts, workspace operations, and activity. A nine-source case proves bounded inspection while the existing worker verifier still accepts all nine valid sources. - Review follow-up: show allowlisted saved detail beside a changed current observation, preserve all listed actions, and follow exact-action selection in verified/limited/error tests. The prior component fails the saved/current drift regression in an in-memory RED control; the current component passes. UI typecheck/build pass again. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, maps, module boundaries, and `git diff --check` pass. - Rebased onto master `46de15ff4b` without conflicts. Range-diff proves both diagnostic commits are unchanged. On new head `a7e2bc87fcea75e76b6b736304d1287257f71174`, all 23 focused tests, full typecheck/build, token gates, maps, and module checks pass again. Both fresh reviews are 5/5 with zero unresolved threads. Final exact-head GitHub status is CLEAN: 55 success, 2 skipped, 1 neutral, no pending checks or failures. The [ci / verify aggregate](https://github.com/paperclipai/paperclip/actions/runs/40387002325/job/121802415824) passed in eight seconds. All test/build/typecheck/E2E lanes passed. - Existing full workspace recovery suite: 56/56 pass (103.5 seconds) before the byte-equivalent rebase. `pnpm test:run` hit repeated embedded PostgreSQL initialization failure before database-suite assertions and was stopped. No full local suite pass is claimed. The owned external `_test` database provided the focused HTTP/source verification. The prior browser attempt also stalled at Network.enable before its click; that attempt did not establish browser acceptance. - Current-head local browser acceptance passes through the actual run page: Inspect recovery lists the exact action, Inspect current source shows recorded generic failure separately from the current rejected policy, and reload resets inspection to the explicit control. Explicit selection works again after reload. The app used isolated data, an empty inherited environment, a paused synthetic agent, and no provider credentials. Final state remains failed/exhausted with zero attempts, zero wakeup receipts, zero workspace operations, and unchanged original generic failure. The local app and temporary tab were stopped. This proves the diagnostic journey, not the deployed source's actual lease policy or its recovery. - Final merge verification at `a7e2bc87fcea75e76b6b736304d1287257f71174`: GitHub reports CLEAN and mergeable. Both repository-required checks, `ci / verify` and `ci / e2e`, pass, as does the exact-head Greptile check (5/5, zero unresolved threads). The separate [Entire trail](https://entire.com/gh/paperclipai/paperclip/trails/7) has no recorded reviewer approval; no such approval is claimed. It is not a configured GitHub merge gate. - Initial local setup failures were distinct from product assertions: localhost sandbox EPERM required scoped local access; a browser harness import path was corrected. A test UUID type inference error and historical native-export UI selection gap were corrected before the green run. The live task remains blocked; this diagnostic does not claim it recovered. ## Risks - This change improves evidence. It does not prove provider termination, local descriptor integrity, or successful file preservation, and it does not repair the original source. - Current inspection spans multiple database reads. It is an observation, not a historical cause or execution authority. It performs no provider/filesystem calls and starts no retry or executor. - The optional persisted detail is backward compatible. Unknown values and extra fields are excluded. Oversized source lists remain uninspected through this explicit diagnostic path; physical recovery rules do not change. ## Model Used OpenAI Codex based on GPT-6, with code execution and browser tools. The exact serving model ID and context window were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 08:21:28 -05:00
DottaandPaperclip	13709b4c7a	feat(connections): let agents configure connections within delegated authority (#17737) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Connections provide governed tools, chat channels, model accounts, and gateways. > - Several management routes require a human board actor even when an agent has the required grant. > - Default agents must configure these resources within the responsible user's authority. > - This pull request adds a shared authorization path to the existing REST and native API surface. > - Human authentication, execution approvals, and local host boundaries remain enforced. > - The benefit is that a standard agent can complete authorized setup with a bounded human handoff. ## Linked Issues or Issue Description **Subsystem affected** Server authorization, connection services, shared contracts, native API discovery, database schema. **Problem or motivation** Connection administration is available in the web app but often unavailable to agents. Agent grants, responsible-user authority, credential ownership, and actor attribution need separate checks. Opening gateway-token creation also requires preserving the issuing agent's delegated authority during token use. **Proposed solution** Use the existing authorization evaluator for every connection family. Enforce current responsible-user authority, effective trust, grant scope, resource visibility, and target-agent access. Keep existing agent grants unchanged. Expose authorized configuration through default `search_api` and `call_api`. Save human authentication steps and let agents continue after completion. **Roadmap alignment** This change completes the API access path for the existing Connected Apps, Personal & Shared AI Accounts, and governed MCP Gateway capabilities in `ROADMAP.md`. It follows the maintainer's requested design. Related work: #11899 enforces connection execution access and creator/manager rules; this PR extends agent management across connection families. It does not replace that execution-policy work. ## What Changed - Add shared connection management context and mandatory responsible-user, trust, scope, and target checks. - Allow authorized agents to configure tool, chat/email, GitHub/Slack, AI, profile, and gateway resources through existing routes. - Preserve human credential ownership and record the actual agent, responsible user, and run or key in audit data. - Preserve saved browser authentication handoffs and callback session checks. - Bind agent-issued durable gateway tokens to their responsible user and recheck delegated authority at use. - Add explicit REST transport and management capability metadata, built-in guidance, and sensitive-response handling. - Intersect caller and endpoint-agent Slack test/invitation policies, including independent and shared rates. Require the addressed human’s current endpoint identity link before completing identity handoffs. - Add production-authenticated integration scenarios. Companion #17739 contains the saved setup UI (including the required human import/rotation step), product and permission documentation, catalog/Vercel credential and callback ownership, and operation maps to keep each review below 100 files. Both PRs must ship together; merge this implementation first, then its companion. ## Verification - Current implementation head: `2ecd6451d0d0c8d82121a4e3fa4ce94c5befbbd6`, including master `ea727dc3c2`, 99 files. The latest follow-up fixes the two valid findings from the preceding f9 review. Exact-head Greptile has completed at5/5 with no findings and Superagent passed. Full exact-head Linux CI workflow40446606056 completed SUCCESS; only the queued external Paperclip Review remains pending; earlier Linux passes do not qualify this successor. - Latest preserved-stack verification passed **2,165 tests** in fresh PostgreSQL: **70 actual-auth scenarios**, 27 OpenAPI contracts, 2,022 native API tests and 46 saved GitHub wizard cases. **141 existing gateway/policy acceptance tests** also passed. Workspace typecheck/build, maps and token gates passed. The first workspace typecheck was blocked by sandbox tsx IPC permissions; its authorized rerun passed. This is focused local evidence; the full Linux CI matrix is still required. - Issuer/target gateway policy regressions were red before the fix. Both decisions now intersect, including Block/Ask first/rates, same credential identity and current token/user/agent authority after approval. Invocations retain their token and actual requesting agent; deleted tokens fail closed. Thirteen actual-auth cases cover revocation, narrowed scope, approvals, shared rates and board compatibility. Intermediate missing-token attribution, review-context and implicit-session actor failures remain in the qualification record; all final cases pass. - Activity pagination now retains a bounded cursor across private-only windows. The actual-JWT regression reaches three older readable events after 27 private ones without exposing private details. A native device-login regression was red for plaintext receipts; local check and test-call results now use existing protected-response handling. The passing actual-auth case proves encrypted persistence, exact replay, redacted history and no response asset. - Previous-head `5b55195593` Linux CI initially failed e2e shard 4/8 in the existing connection-intent continuation test: periodic recovery queued a repair at 01:21:47, the scripted journey completed the task at 01:21:48, and the extra queued run was cancelled at 01:21:49 because the task was terminal. The assertion selected that latest cancelled run. GitHub refused the first attempt while the workflow was nonterminal. After completion, the single evidence-supported failed-job rerun was accepted as **attempt 2** on the same head (workflow 40431202466). The failed shard and final E2E aggregate passed on attempt 2, and the complete Linux workflow for that head is successful. The original failure remains part of qualification. No additional unchanged-code rerun will be used for this failure. - Companion CI also has an unchanged login-cleanup idleness timeout, retained in its qualification record. That same 16-case suite passed in this parent Linux run and a fresh local database. Both preceding workflows passed after their single failed-job reruns as attempt 2; the original failures remain recorded. No additional unchanged-code rerun was used. - Migration `0380_broken_aqueduct.sql` was regenerated through the repository workflow after master took 0379 for project Git authorization. It adds the responsible-user column with `IF NOT EXISTS`, preserves upstream migrations, and satisfies snapshot retention and migration safety checks. - Upstream published Copilot v62 and v63 identities remain byte-for-byte unchanged. The combined guidance is now v64, digest `sha256:22e31f95ef972bcead98a4a707c9f33433558fd111e390e027a2bbb4346e112a`. The normal shared TypeScript/Rust identity check and **61 Copilot installation tests passed**. The old source-identity assertion first failed against unchanged published v63; regenerating a new v64 preserves the published release. - Workspace `pnpm -r typecheck` and `pnpm build`, map validation, and UI token gates passed on the final combined stack after the upstream merge. Node 24.22.1 / pnpm 9.15.4 were used. - **Earlier 82 API/OpenAPI tests passed** in fresh PostgreSQL 18: 55 assembled-server scenarios and 27 contracts. They use real persistent agent keys, run JWTs, human sessions, production defaults, and deterministic provider fixtures. No actor injection establishes access. - The new email regression was red before per-account scope filtering. The Arcade regression was red before human ownership and inventory-key fixes; it also verifies actual-agent audit identity and a human-session inventory read. The activity regression measured **200 authorization checks before the fix and one afterward** for the same one-item page. - **82 GitHub UI tests passed**, including all three new failed-replacement regressions that were red before their fix. The UI preserves an earlier copy only after confirming its committed generation. Changed or unreadable results block copying and reconnecting until reconciled. Human-only secret responses now include their nonsecret committed generation. - Private test status now rechecks the originating principal and linked task, including mandatory agent/responsible-user privacy. Cross-agent tests require the full caller tool access decision and the same credential identity, regardless of auth label; caller and target policies/approvals and rate limits apply before execution and after human approval. A profile-only precondition was found to reject valid allow-policy and scoped-grant access and has been removed. Both permitted-access cases and a no-auth-label identity regression were red before the correction; caller-specific Block still prevents provider execution. The private-status regression was also red before its fix. - The tool-activity regression measured 100 capability decisions and 102 run captures on a 100-event page before the fix. It now uses two task decisions and at most two captures while retaining the same 50 visible events, private-event filtering, and 403 after audit-grant revocation. The regression was red before request-context reuse and task-decision sharing; final server typecheck/build also pass. - Generic chat configure/reconnect now hands agent-supplied verification credentials to the addressed human before provider work or vault changes. Actual key/JWT regressions were red for Slack signing-key replacement and generic channel secret-reference rebinding. They now prove unchanged refs and vaulted signing value, human setup, permitted agent bot-token repair, and governed test messages. All 82 API/OpenAPI cases pass in fresh PostgreSQL. Eight native receipt protections and final server typecheck/build also pass. - The preceding review found that old healthy state could retire a credential-entry card before a human changed anything. The actual-JWT regression was red. Saved field generations now change only with protected values and commit with refs plus a verification fence; retries retain the baseline. Old health, bot-token repair and unchanged entry stay pending. New signing verification and current authority permit continuation. A separate red race proved that user revocation during provider authentication could return 403 after refs changed; authority is now checked before persistence and inside the credential transaction. The final 82 API/OpenAPI cases, four neighboring real-database Slack/GitHub lifecycle cases, and final server typecheck/build pass. - A preceding full run crossed a rate-window boundary. Database rows prove one consumed slot at 23:35 and another at 23:36 for the same policy/bucket. Only Date is frozen in the rate test cases; transport and asynchronous timers remain real. Production rate behavior is unchanged, and the final full run passes. An intermediate fixture failure that overwrote hidden generation metadata with an API projection is retained in the qualification record. - Linux CI on an earlier head found two rate-admission assertions that omitted the newly persisted bucket key. The failures were reproduced locally; both assertions now verify the exact actor/tool bucket and all 49 policy-service tests pass. The companion qualification record retains the initial and intermediate failures. Catalog creator attribution is preserved in the companion to keep this implementation review at 99 files. - The latest review reproduced three regressions before repair: complete human replacements failed on unavailable old signing secrets, fresh GitHub links incorrectly selected reconnect, and different credential-entry requests shared an idempotency key. Each old protected ref now has a narrow known-unavailable fallback; unexpected resolver failures and company/scope violations still abort. Fresh imports keep their credential form, configured endpoints keep reconnect, and canonical field sets/completion modes distinguish handoffs while matching retries retain their baseline. All 55 actual-auth scenarios and 27 separately run OpenAPI contracts pass, as do server typecheck/build. Companion #17739 also passes all 41 wizard tests including actual fresh-form submission, UI typecheck, maps and token gates. Earlier red results and a mistyped OpenAPI filename are retained in qualification. - The configured GitHub handoff regression was red in the browser-component tests and actual-auth link/persistence tests. Saved links now name requested fields and all/any completion mode; the companion form submits those supplied fields through reconnect and retains other saved credentials. Webhook/client-secret reconnect remains human-only; first-setup secret restrictions remain. All 46 wizard tests pass, including combined modes and legacy links. A duplicate review referenced the older companion bf760912; the current form is already fixed, linked in the reply, and its thread is resolved. - Linux CI on bf760912 failed automatic Slack provisioning because vault persistence cleared an already verified manifest-update challenge. The same regression was reproduced locally. The provider-owned helper now preserves proof only for the current signing fingerprint and callback URL. Eight existing compatibility cases pass, including this case and initial GitHub secret rejection. The first uncommitted GitHub-field variant also failed that initial boundary and was narrowed to reconnect; both failures remain in qualification. - The upstream merge conflicted only in generated migration metadata. Upstream 0379 is preserved, normal generation/pruning produced 0380, and two repeated applications of the idempotent additive SQL passed. The final combined stack passed **128 tests** (55 actual-auth, 27 OpenAPI and 46 wizard cases) in fresh PostgreSQL. Eight Google UI cases also pass; the two protocol cases could not initialize embedded PostgreSQL in this fresh local attempt. Their earlier passing evidence remains historical; fresh Linux CI must execute them. Vercel personal credential attribution is preserved in companion #17739 with its other ownership/consent follow-ups, keeping this PR at 99 files. - The suite covers fresh defaults, restricted users/agents, scopes, personal isolation, token revocation, human handoffs, callback replay, execution approvals, authorized self-targeted policy changes, self-approval denial, and current authority before commit. Reproduce API checks with `pnpm exec vitest run server/src/__tests__/connection-management.integration.test.ts server/src/__tests__/openapi-routes.test.ts`, using a fresh database through `PAPERCLIP_CONNECTION_MANAGEMENT_TEST_DATABASE_URL` or the default embedded test database. - Companion #17739 also records consent separately from token refresh so saved Google choices survive refresh. Its eight service-picker and two local-HTTP OAuth tests pass; both new UI cases were red before the fix. Normal migration generation confirms no SQL change is needed for the additive JSON metadata. - The integration module shares one assembled server, authentication factory, and provider lifecycle to preserve the HTTP-to-vault-to-task-continuation proof. A reused external database caused a Slack fixture identity conflict; the fresh final run passed. Earlier fixture/transport and contract-update failures remain in the companion qualification record. - The broad local `pnpm test:run` attempt remains blocked by macOS embedded PostgreSQL's host semaphore limit. It is not a clean pass. The fresh Linux CI matrix must complete. Provider fixtures do not prove live setup or an empirical LLM success rate. - A current-head completed Paperclip assessment is not yet recorded. Its earlier external recovery boundary has not been bypassed, and no incomplete assessment is reported as passing. The user authorized both merges when checks pass; merge this implementation first, then replay, retarget, and requalify companion #17739 against the squash-merged master. - After the master refresh, local workspace typecheck/build, **97 actual-auth/OpenAPI cases** and **141 gateway/policy cases** pass. The original fixed database port was occupied; Docker assigned a free localhost port. The first neighbor setup command could not resolve root `tsx`, leaving its schema uninitialized; the built database package migrated the separate fixture and all unchanged cases passed. Both setup failures are retained in qualification. Fresh Linux CI remains required; no full local test-sweep pass is claimed. - The f9 Greptile review scored **2/5** for two reproduced findings: Slack test sends skipped the caller’s tool policy, and healthy App credentials completed an unconfirmed identity handoff. Both are fixed in this head and have resolved evidence threads ([Slack](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927765), [identity](https://github.com/paperclipai/paperclip/pull/17737#discussion_r4384927895)). Actual key/JWT cases cover twelve Slack caller/target/recipient combinations; the human-session GitHub OAuth callback alone remains pending until explicit confirmation. All **99 API/OpenAPI tests** (72 actual-auth + 27 contracts), workspace typecheck and build pass. On the combined companion, all four focused cases (including the twelve Slack combinations), typecheck/build and maps also pass. The continuation suite’s 50 embedded-PG cases were skipped because its database could not start locally; fresh Linux CI must run them. Initial fixture credential-collision and audit-projection assertion failures are retained with the red reproductions in the companion qualification record. No full local sweep or live consent is claimed. ## Risks - The maintainer explicitly confirmed authorized policy administration, including self-targeted changes that loosen controls. These writes require management grants and the responsible user’s current authority. Agents cannot approve their own governed action requests or edit their own principal permission grants. The actual-auth regression exercises both permitted administration and those denials. - This is a broad authorization change. Incorrect scope or owner resolution could deny permitted setup or expose another user's resources. Integration cases cover both permitted and denied requests with actual authentication. - The additive nullable gateway-token column does not infer identities for legacy clients or backfill agent grants. Legacy agent tokens without a responsible user fail closed. - Provider fixtures test supported setup and callback contracts. They do not prove live provider availability. ## Model Used - OpenAI Codex, GPT-6 family, with reasoning, repository tools, and code execution. The runtime does not expose a more specific deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — focused suites above; full local sweep remains blocked as documented - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green — exact `2ecd6451d0` Linux CI completed SUCCESS; external Paperclip Review is still queued - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — exact `2ecd6451d0` review completed; both f9 findings have resolved evidence replies - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 08:20:24 -05:00
Nicky Leach	baec3abd67	Authorize background project Git with explicit Connector consent (#17798)	2026-10-26 17:22:49 -07:00
DottaandPaperclip	5061ab7b35	perf(ui): speed up task and inbox cold opens (#17756) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Task lists and inboxes are the main entry points to that work. > - Cold opens fetch overlapping task lists, task descriptions, and run details before useful rows settle. > - Independent responses can insert rows above content that a person has started to read. > - This pull request loads a small task index first and adds details only for visible rows. > - It keeps row order and geometry stable while those details arrive. ## Linked Issues or Issue Description **What existing behavior does this improve?** Cold opens of Tasks and Inbox, including the shared navigation badge. **Subsystem affected** Server list reads, shared response contracts, and the React UI. **Current behavior** The badge fetches full task bodies. Mine fetches several overlapping lists. Historical diagnostics and row metadata add work before the first useful list. A fast failed-run response can appear before task rows, then move down when tasks arrive. **Proposed behavior** Load the selected inbox source and its required context together. Use an index without task descriptions or recovery diagnostics. Fetch diagnostic details for up to 50 visible rows. Keep skeletons, recovery indicators, external status, and live metadata in fixed space. **Reason and benefit** Reduce work on a cold open and prevent loading-induced row movement. On the same 100 local fixture tasks, the compact response was 262,441 decoded bytes and the index was 110,909 bytes. This comparison excludes compression and is not a page latency benchmark. **Breaking changes** Existing full and compact responses remain available. New index/row projections are opt-in and marked as partial data. Mine keeps its existing most-recent-100 display limit, now selected on the server. Existing filters and group preferences remain in the client. No database migration. **Additional context** Refs: #17593. Related: #13095, #17428, #13803. This change addresses collection overfetch and layout stability. It does not replace the blocked-inbox query work or the database activity-index work. ## What Changed - Add authorized index and row projections, with bounded task and parent ID batches. - Replace badge list downloads with unread IDs and aggregate counts. Preserve actor privacy, dismissals, and local archive behavior. - Load only the selected inbox source, plus bounded parent and failed-run context. Return only relevant failures from the bounded run window. - Batch hierarchy reads, including at most one authorized child per collapsed parent in one request. Collapsed markers do not consume the expanded row budget, and partial descendant counts are not shown as totals. Enrich visible rows without replacing membership, ordering, titles, or read state. - Reserve row geometry on desktop and mobile. Reveal the initial inbox only after its sources settle. Preserve legacy task-list reads. - Start collection module downloads during access checks. Keep partial rows out of the task-detail cache. - Keep ETags and concurrent request sharing for the new projections, but skip the short server response TTL so archive undo sees fresh state. ## Verification - On final head `ef6afb2a56`, the full CI workflow passed, including build, typecheck, server/workspace/runner suites, and all eight browser shards. A server shard reported **1,489 passing tests** before runner shutdown canceled the job; its single infrastructure retry passed. Both Greptile and Paperclip Review are **5/5**, with zero unresolved review threads. - The optional external **Entire Gates** check remains `unassessed` because it has no active assessed Change for this comparison. Entire is not set up in this checkout. The repository ruleset requires `ci / verify` and `ci / e2e`; both passed. No gate configuration was changed. - Passed workspace typecheck after the final API batching change. The full production build also passed on the final commit; the exact-retry E2E run built the shipped UI. - Passed token gates and map validation. The final batch-preview pass has 91 passing route/hierarchy/component tests, including privacy, request bounds, 30 collapsed parents in one read, and a 120-child parent alongside a sibling. - Passed the 166-test focused UI/inbox/badge run, 47 additional UI regression tests, and 34 issue-list route tests. The latest regressions cover low-trust agent badge IDs, cold company selection, 30 description-only Mine matches with archive actions, retrying optional row details, and collapsed/expanded siblings. - The full local `pnpm test:run` spanned the rebase and reported failures in five suites before it was stopped; it is not claimed as a complete pass. A fresh serial rerun of all five suites passed **452/452** tests (tool gateway, native resume, project repositories, workspace recovery, and runner selection). Full current-commit coverage is checked by CI. - Real browser: delayed index and row-detail responses on Tasks and Mine. The first placeholder and first row had the same origin and 44-pixel height. Twelve rows on each page had zero position, height, or title-width change after recovery details arrived. - Real browser: reviewed long titles and recovery status at a 390 by 844 viewport. Task text stays on one line and recovery uses a compact indicator. - Three fresh document loads per page on final commit `260893bf92`, with HTTP cache retained on a warm local development server: Tasks 1.031–1.190 seconds; Mine 0.999–1.193 seconds. Browser-control overhead is included. These values must not be compared directly with staging latency or the earlier local samples taken during a full test run. - The browser archive test exposed stale results after Undo. The new projections now bypass the server TTL; immediate-mutation regression tests pass. The final browser retry passed: Archive followed by Undo restored exactly one row and the badge count. - CI exposed a hook-order crash when company selection resolved after the first inbox render. The hook now runs before the empty-company return. All seven affected exact-retry and legacy-continuation browser scenarios pass locally after the fix. - Real browser review regressions: 97 description-only source matches retained Archive actions; deliberately failed optional row reads displayed a retry notice while all 12 observed row positions and heights stayed unchanged. Retry restored the recovery indicator. Network fault injection was removed. - The final production build and workspace typecheck passed. Final focused verification passed 153 UI/helper tests and 26 heartbeat privacy route tests. Final review regressions cover newer diagnostics arriving during the index read, reloading diagnostics when the same task ID changes version, optional source failure/retry without hiding available rows, and required external-status filter failure. The latest-failures route test verifies bounded selection, agent filtering, success suppressing an older failure, and private-run redaction. - Real browser: deliberately failed the run source, then retried it. Twelve available task rows kept the same position and 44-pixel height before, during, and after retry. No loading skeleton replaced them. Fault injection was removed. - Hierarchy review decision: the cold-open barrier stays because inserting ancestors or expanded children after revealing index rows would cause layout shifts. Existing hierarchy context now survives subsequent requests, with a regression proving a new ancestor read does not blank the list. The bounded recursive tree read can still add latency for unusually deep expanded trees. - Tasks now handles a failed external-status filter with an explicit error and Retry, suppressing a false empty state. All 52 Tasks component tests pass, including failure-to-success retry. One existing large-tree test timed out while workspace typechecking ran concurrently; the whole component suite passed when rerun without that competing work. Workspace typecheck, the full production build, and token gates passed for this fix. ## Risks - Saved filters and grouping still operate on the existing bounded collection. This is not a new global server pagination model. - Fixed row slots reduce available title width. Long titles remain accessible through task navigation. - Recovery and related diagnostics arrive after core rows. Their responses cannot replace ordering fields, and stale versions are rejected. - Index/row requests now revalidate against current data instead of a short server TTL. They still share concurrent reads and use ETags. - This has not been deployed by this pull request. Staging latency must be measured after deployment under comparable conditions. ## Model Used OpenAI Codex, GPT-6 family. The exact runtime model ID and context-window size are not exposed in this session. Used code editing, terminal execution, and real-browser diagnostics. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 14:29:20 -05:00
DottaandPaperclip	a1f77821b9	perf(tasks): show recent conversation before historical activity (#17593) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task page must show the instruction and current conversation quickly. > - The current page waits for related task data and historical run transcripts. > - These reads grow with task history, even when the reader only needs the latest answer. > - This pull request gives cold opens a small initial view and loads older history on demand. > - Messages and runs observed while the page stays open remain visible. > - The benefit is a readable task before historical diagnostics finish loading. ## Linked Issues or Issue Description Related: #14667 and #16229. The first changes the reveal gate and scroll anchors. The second adds general history pagination. This PR adds a core task read and a folded recent-conversation view. It preserves the default full-read API contracts. The shared files will need coordination if either related PR lands first. **What existing behavior does this improve?** Cold opening an ordinary task with a long conversation or large run logs. **Current behavior** Task enrichment and historical transcripts delay the readable conversation. A cold open can fetch several megabytes of tool activity before revealing saved messages. **Proposed behavior** Show the task instruction, recent saved replies, latest human input, and the latest run's conversation data first. Fetch older messages and full run activity when the reader chooses **More messages**. Retain new messages while the page stays open. ## What Changed - Add an authorized core issue view that skips related enrichment. Keep enrichment on the existing detail read. Share task mutation, rollback and invalidation updates with the core cache. - Add a bounded recent-comment read. Preserve the first human instruction when no description exists, including legacy rows with a NULL author type. Keep company and task access checks and comment redaction. - Add optional recent activity and run-summary views. Fetch original input receipts for visible comments separately so delivered messages stay out of the queue. Reconcile watched unfinished runs after a reconnect beyond the preview window, removing deleted runs after reconciliation. - Filter the latest native run's initial events to assistant messages, request lifecycles and terminal results. Full event reads remain available on expansion. Legacy fallback reads use a small tail extended to the first complete record. HTTP cancellation aborts the storage read. - Retain observed messages across cursor catch-up and new runs. Apply local cancellation and deletion to both caches and their task aliases. Refetch locally seeded history pages on expansion. Preserve expansion across identifier and UUID links. Keep deep links, classic tasks and Agent Chat on their existing full-history path. - Update API documentation, design guidance, capability maps and regression coverage. ## Verification - Final acceptance on `9ea0b15ae44d9f6d2c9a9f824b4835f299f92979`: all CI gates passed, including server/UI tests, all eight browser shards, runner checks, full typecheck and build. [Final CI run](https://github.com/paperclipai/paperclip/actions/runs/40326990328). GitHub reports 55 successful checks, two intentionally skipped checks, and one neutral Paperclip Review check described below. No checks are pending or failed; Greptile is 5/5 and no review threads remain open. - October 26 merge update: merged current `master`. The only conflict was in task-page imports; both the compact-history hook and the new live-update hooks remain. All CI tests and build jobs passed on `044fd6733d`. Fresh review found canceled-comment cache drift, legacy human-comment filtering, send-then-expand truncation, and partial legacy log records. These defects were fixed in `af3d66a5bd`. - Review-fix verification: 470 tests pass across ten affected suites, including all six PostgreSQL cold-open tests without a helper override. Cancellation/delete and send-then-expand regression tests failed before the fixes and pass afterward. Coverage includes task identifier/UUID reopen, stale in-flight reads, legacy NULL author types, local-file and object-storage tails cut inside UTF-8 text, route forwarding, and request cancellation. Full workspace typecheck and production build, token gates, map validation and diff checks pass. - Follow-up review found that a failed expanded legacy read lost its record-alignment state on retry. `9ea0b15ae4` commits the expanded window only after a successful read. The new regression fails before the fix and passes afterward; all 19 transcript tests, UI typecheck, UI build and token gates pass. Greptile reviewed this commit at 5/5 with no actionable findings, and all review threads are resolved. - The final Paperclip Review check ended neutral because its execution did not complete; no passing Paperclip assessment is claimed. Its earlier four concrete findings were fixed and regression-tested. Greptile and the security scans completed successfully on the final commit. - The preceding `af3d66a5bd` CI run was canceled when superseded by the retry fix. Its server shard 11 check briefly reported failure, but the job API records cancellation and its test step passed all 1,750 tests. This superseded run is not used as final CI acceptance. - 549 focused tests pass on the final implementation across nine task UI, activity-route, OpenAPI and PostgreSQL suites. These cover pending history, desktop/mobile visibility, expansion, retained runs, message bursts, cancellation recovery and queued-comment removal in both views. - Legacy expansion keeps the latest answer visible while fetching and deduplicating older output. - Five PostgreSQL tests pass for core access denial, recent comments, missing-description instructions, event paging and bounded activity/run reads. macOS could not start another PostgreSQL cluster because it exhausted its semaphore limit. These tests used a separate disposable database on an existing local server through a temporary test setup override. The helper was restored afterward. - Full workspace typecheck and build pass with the final state fixes, as do token gates and capability-map validation. - Token gates, capability-map validation and `git diff --check` pass. - Browser proof used a disposable task with 80 comments and six large run histories. The cold view showed the instruction and recent exchange. **More messages** revealed older comments. Navigation away and back preserved expansion. - Local HTTP sample: core 50 ms, recent comments 24 ms, recent run summaries 17 ms. The latest native conversation response was 839 JSON characters / 8 ms; its full event response was 1,099,005 / 86 ms. Browser content paint was 1.34 seconds with development assets cached. These are local observations, not staging latency claims. - The first CI run passed every job except four board-send browser cases and the review threshold. Those browser fixtures only mocked the old comments endpoint; they now serve both recent and full reads, retaining all message and attachment assertions. Both initial review findings were fixed with regression tests. The next CI run passed every job. The deeper review identified core-cache mutation, older delivery receipt and retained run-status gaps; those are now fixed with regression coverage. Final review is 5/5 with zero unresolved findings. Final-head CI initially failed one unchanged runner shutdown test (`runnerd-codex-transport.test.ts`, timeout-attachment case) with `NativeSessionCloseUnrecoverableError`; all other CI jobs passed. The same runner code passed on 971a762f80. A targeted local reproduction also failed during provider drain. The failure is outside the changed task loading paths; its underlying cause remains unresolved. The single failed-job retry passed on the same head; this demonstrates recovery, not a root-cause fix. All CI gates passed on the previous implementation head `9004f94f28`: [CI run and retry](https://github.com/paperclipai/paperclip/actions/runs/40192334629). - `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL startup failures. The native session suite recorded 15 database-related failures; other database suites could not start. No complete local suite pass is claimed. ## Risks - The compact view intentionally omits older diagnostics until expansion. Existing open questions and approvals still use the current interaction and run paths. - Previously observed comment and activity rows remain cached while open. The newest preview and incoming cursor pages refresh; this does not add a general historical-edit synchronization protocol. - Recent views are optional. Existing full API callers retain their response shape and behavior. No migration is required. - A legacy tail can exceed the requested byte limit by the prefix of its first log record. Backward reads stop at the preceding record boundary and honor cancellation. - Staging has not been deployed or qualified with this change. Instance startup latency is separate from task history loading. - The related open PRs touch the same task-history files. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository editing, shell tools and browser testing. The exact serving model identifier and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — 549 affected task tests; the full local suite and unrelated runner failure are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-26 08:11:39 -05:00
DottaandPaperclip	d0c8f0b090	fix: explain project access denials and add an authorized repair path (#17509) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Low-trust agents may read only the projects in their allowed scope. > - Workspace setup correctly stops a task outside that scope. > - The task previously showed a generic failure, and agent settings did not offer an additive repair. > - This pull request explains the denial and lets authorized operators add projects while preserving other restrictions. > - The saved task then offers an explicit retry. Saving access does not start work. ## Linked Issues or Issue Description **What happened?** A project access denial appeared as an ordinary failed attempt. Operators could not add a second allowed project from agent settings. **Expected behavior** The task should explain the denied access and link to a working permission editor. The editor should preserve existing scope and restrictions, enforce operator authority, and retain drafts after a failed save. **Steps to reproduce** 1. Assign a low-trust agent to a task whose project is outside its allowed projects. 2. Let workspace setup deny project access. 3. Open the task, follow Manage project access, add the project, save, reload, and return to the task. 4. Confirm that the saved message remains and Retry task is a separate action. ## What Changed - Record a typed, non-retryable project denial before provider dispatch. Project denials do not enter automatic continuation or provider comment policy. - Project current and historical setup denials into a safe task notice. Keep unrelated failures, prior assignees, and approval/pause/budget gates separate. Recheck current project authority, including standard trust and root-issue grants. Accepted retries keep their working state. - Add board-only project-access reads and additive saves under the agent row lock. Enforce company scope, agent-management authority, pending-approval restrictions, and an audit record for new additions. - Add the production notice and searchable multi-project editor, including read-only, saving, cancellation, and failed-save states. Preserve the return-to-task query through canonical agent redirects. - Register board-only project-access routes in OpenAPI with a strict request schema. - Add production Storybook examples and API/persistence/concurrency regressions. Retain the approved design snapshot and update the behavior maps and implementation contract. ## Verification - Passed: `pnpm -r typecheck`, `pnpm build`, `pnpm build-storybook`, `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `node scripts/storybook-catalog.mjs`. - Passed: 11 permission/projection tests and 30 run-completion tests against an isolated PostgreSQL database. These cover historical failures, concurrent additions, idempotent audit, preserved restrictions, foreign projects, read-only users, agent self-grants, and no automatic wakeup. - Passed: all 27 OpenAPI route tests, including route coverage and the new board-only request/response contract. - Browser: the real app displayed a historical denial; Manage project access opened the editor; saving retained the original project and survived reload; Return to task showed the ready notice. Saving created one audit event and zero runs. Explicit Retry returned 202 and reached the separate sandbox requirement, which remained enforced. No external provider was used. - All five production Storybook interaction checks pass: blocked task, save, cancellation, failed-save recovery, and read-only settings. These use fixture callbacks; durable authorization is covered separately above. - `pnpm test:run` was attempted locally but stopped after repeated embedded PostgreSQL startup failures caused by macOS semaphore exhaustion. Full-suite CI is required before merge. The separate UI suite passed all 9,095 tests. The 41 focused server regressions also passed after rebasing onto the latest master. A further 26 recovery-classification tests pass, including a red-to-green regression for automatic continuation after a project denial. ## Risks - No schema migration. The new endpoint only adds same-company project IDs and preserves the rest of the stored policy. - The historical notice recognizes one exact setup denial message. It does not display arbitrary run errors or disclose a different source project's identity. - The retry still passes all current admission and isolation checks. The notice does not promise that other execution requirements are satisfied. - Related open work: #13309 covers inherited secret restrictions; #11234 covers the broader repository catalog. This change does not replace either policy surface. ## Model Used OpenAI Codex (GPT-6), with code execution and browser tools. The precise runtime model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused server and full UI suites; full-suite infrastructure limit above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 16:16:29 -07:00
DottaandPaperclip	0eb2c964a7	fix(agents): allow runner qualification with agent configuration permission (#17448) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent owners configure the harness and computer used for that work. > - A legacy runner upgrade must first qualify an unverified sandbox. > - The upgrade called an instance-admin probe, so a company owner with agent configuration permission received HTTP 403. > - This pull request qualifies the saved computer through the agent's existing configuration authorization. > - Company owners can complete the upgrade while general instance environment management stays restricted. ## Linked Issues or Issue Description Refs #17361. Related: #17122. **What happened?** Upgrade runner displayed for an existing Claude agent on a Daytona Paperclip Computer. Clicking it returned “Instance admin access required” for a cloud company owner. The agent remained on the legacy runner. **Expected behavior** A caller permitted to update the agent can qualify that agent's saved computer and upgrade. Instance-admin access is not required for this agent-scoped operation. **Steps to reproduce** 1. Sign in as a cloud company owner with `agents:configure` and no instance-admin role. 2. Open an existing legacy Claude agent on an unqualified Daytona snapshot. 3. Select Upgrade runner. ## What Changed - Add `POST /api/agents/{id}/qualify-runner` with the existing `agent_config:update` guard. - Reuse the server's effective environment resolver. The submitted environment ID only detects stale state; it cannot select a different computer. - Preserve company binding checks, check binding again at lease acquisition, and record agent-scoped audit activity. - Return qualification status and the lease receipt without instance-level provider metadata or raw provider errors. - Use the new endpoint for the existing-agent upgrade. Keep the generic environment probe restricted to instance admins. - Disable Upgrade runner when an unsaved computer change resolves to a different target, with a save-first explanation. Other draft settings still accompany an upgrade. - Add real-database authorization regression tests and update the API contract and access maps. ## Verification - Red: the existing UI called the instance-wide probe and failed the new agent-scoped call assertions in eight upgrade cases. - Green: 349 related route, provider qualification, runner selection, and UI tests passed. - Green: 12 authorization cases passed with real PostgreSQL, real HTTP routes, real saved role grants, and a simulated provider boot. They cover the cloud owner, removed grants, foreign agent/computer, inherited targets, stale target, invalid body, archived/local target, redacted failure, audit scope, and unchanged agent settings. - Review regression: two cases first reproduced the enabled upgrade button after an unsaved computer change; both now pass, along with all 168 related UI tests. Qualified/unqualified computers and restoring the saved selection are covered. - All 26 OpenAPI cases passed; the 12 real-database authorization cases also passed after rebase. - The host exhausted PostgreSQL semaphores. For the 12 cases, a temporary local-only setup used disposable PostgreSQL 18 in Docker instead of the embedded database launcher. The committed tests use the normal embedded database helper. No authorization decisions were mocked. - `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, map validation, module-boundary validation, and `git diff --check` passed. - `pnpm test:run` was attempted but did not establish a full local pass: the host could not start embedded PostgreSQL clusters. All current-head CI gates passed, including the full suite in CI. The commitperclip review workflow timed out before reporting any executed steps on its first attempt and passed on one retry. - Fresh staging acceptance remains pending deployment. Verify persisted native configuration and a successful runtime test on the existing Daytona computer after merge. Earlier live Daytona proof from #17361 is separate evidence. ## Risks - Qualification can start a fresh, temporary sandbox. It uses the existing probe lifecycle and cleanup contract. - The new path must remain scoped to the saved agent and its company. Denial and redaction tests cover that boundary. General instance probe authorization is unchanged. - No migration, role change, credential change, or automatic agent upgrade is included. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model variant and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 10:31:40 -07:00
DottaandPaperclip	87bf89756e	fix: support Daytona runner upgrades and ignore obsolete Claude options (#17361) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Existing Claude and Codex agents can upgrade to Paperclip Runner. > - A Daytona computer needs platform qualification before Runner becomes its default. > - The upgrade notice required that default, so it hid the action that could start qualification. > - Saved Claude turn limits also caused conversion to fail even though Runner can run without them. > - This pull request verifies the computer on click and ignores obsolete Claude options with a warning. > - Agents retain their model, account, environment, instructions, and permission policy. ## Linked Issues or Issue Description Related: #17122. **What happened?** A legacy Claude agent on a Daytona snapshot had no upgrade button until its environment had platform qualification. A saved `maxTurnsPerRun: 1000` then caused conversion to return HTTP 422. **Expected behavior** The upgrade action verifies a supported Daytona computer and converts the agent. Unsupported tuning options produce a visible warning and do not block execution. **Steps to reproduce** 1. Create a legacy Claude agent with `maxTurnsPerRun: 1000` on an active Daytona snapshot. 2. Open Runtime before the environment has platform qualification. 3. Try to upgrade to Paperclip Runner. **Paperclip version or commit** After #17122, reproduced against `9c442cf48e`. ## What Changed - Offer upgrade on probe-capable Daytona snapshots and immutable sandbox images. Perform the public environment probe only after a click. Save after fresh adapter discovery confirms support. - Resolve the actual computer for explicit, instance-default, and hidden managed environments. Preserve pending edits and reject stale probe results. - Ignore Claude `maxTurnsPerRun`, `chrome`, `warmHandleIdleMs`, and `acpWarmHandleIdleMs`. Return structured warnings from create, hire, and update. Show warnings in Runtime and after a save. - Keep validation for commands, credentials, filesystem, network, and permission settings. - Retain the legacy turn limit in the existing live conversion fixture. Update regression tests, upgrade stories, and capability maps. ## Verification - Red: the old route returns HTTP 422 for the saved turn limit; the old form hides the Daytona upgrade notice. - Green: 417 focused server and form tests pass. A further 188 tests pass after the review fixes. Qualification coverage includes failed probes, unsupported targets, changed settings, managed defaults, unsaved environment variables, and forced Kubernetes targets. - `pnpm -r typecheck` and `pnpm build` pass. Token gates, map checks, map tests, and Storybook catalog checks pass. - Real local Claude Opus 5.5 conversion and task pass, with the downloaded artifact checked against seeded source bytes, SHA-256, and Git commit. - Real browser upgrade and full Claude Opus 5.5 task on a disposable named Daytona snapshot pass at `b81cec7d8c`. The saved agent retains its model, high effort, credentials, timeout, and grace settings while dropping the turn limit with a visible warning. Exactly one successful run completed the task. The downloaded artifact matched independently seeded source bytes, SHA-256, and Git commit; the source was unchanged, no interactions remained pending, and the delivery link survived a browser reload. The snapshot used current runtime assets built from public source. The supplied Daytona account could not access the original staging snapshot, so this verifies an equivalent local fixture rather than that exact snapshot. Disposable test resources were cleaned up. - Earlier live attempts remain recorded: an image-based sandbox creation failed, and the first named-snapshot probe returned failure with unresolved provisional cleanup identity. Later named-snapshot probing and the task succeeded; the initial creation failures are not attributed to this PR or declared resolved. A separate attempt using the Oct 18 cached runtime failed the current provider-pack compatibility check; rebuilding the runtime from matching public source removed that mismatch. Owned resources were identified and cleaned up, with diagnostic databases retained. - 54 checks passed at `b81cec7d8c`, with no failed checks; Greptile and Paperclip Review both scored 5/5, with both review threads resolved. Master advanced during the live proof, so the branch was rebased with both sides of additive import/test conflicts retained. On the rebased head `2b74222b94`, 472 focused tests, UI typecheck, and UI build pass. All required CI gates pass; Greptile and Paperclip Review both score 5/5 on this head, with no unresolved review threads. The PR is mergeable. The serial local `pnpm test:run` was stopped after the CI test lanes passed; that local run is not claimed as a pass. ## Risks - An explicit upgrade can allocate a temporary probe sandbox and take several minutes. Probe failures leave the agent on its existing runner. - Ignored options no longer enforce a turn cap, Chrome integration, or legacy idle-time tuning. The UI warns about each active ignored option. - Unsafe or ambiguous execution settings still fail validation. This change does not weaken environment qualification or runtime artifact checks. ## Model Used OpenAI Codex, GPT-6, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 07:59:18 -07:00
Nicky LeachandPaperclip	6204239cfc	Support installed default project workspace providers (#17247) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Storage plugins already own project storage and task checkout preparation. > - Core only selected these plugins through an explicit project policy. > - Installation could therefore leave new and existing projects on incompatible Core preparation. > - This pull request adds one generic manifest declaration and one shared effective-policy resolver. > - Project configuration shows inheritance while durable workspace bindings retain their owners. ## Linked Issues or Issue Description **Subsystem affected** Plugin contracts, project configuration, and execution workspace preparation. **Problem or motivation** An installed storage provider cannot become the company default. Projects without an explicit provider keep Core preparation, even when their execution environment cannot use a Core Git checkout. **Proposed solution** A provider declares supported `workspaceProvider.defaultForProjects` mode and strategy. Company-enabled installations supply inherited defaults. Explicit plugin selections and explicit Core/local (`storageProviderPluginId: null`) win. Conflicting defaults require an operator choice. The declaration and implementation contain no provider-specific identifiers. **Alternatives considered** Rewriting every project would lose the distinction between inheritance and overrides. Selecting by worker health would silently change storage on an outage. **Roadmap alignment** Extends the existing workspace and sandbox provider contracts. The roadmap has no duplicate default-provider implementation. Related public work includes #17243, which covers prepared execution directories rather than provider selection. ## What Changed - Add a validated generic default-provider declaration and document it in the SDK. - Resolve provider selection and Git defaults consistently in project responses, source selection, heartbeat source resolution, reservation, and preparation. - Keep stored policy separate from effective configuration. Add the normal project configuration selector for inheritance, explicit plugins, and Core/local. - Preserve existing local and plugin workspace ownership. Pin selection through preparation and keep provider failures visible. - Add database, API, manifest, heartbeat, and UI regressions. Update the verification maps. ## Verification - Targeted Vitest runs cover manifest validation, project defaults and API access, explicit overrides, existing ownership, new/imported task inheritance, heartbeat routing, source-free tasks, selection pinning during setup, and UI configuration. - Combined policy, provider integration, and heartbeat run: 58 tests pass; 60 unrelated cases excluded. After the final shared-provider fallback fix, all 47 policy tests pass, including shared and operator-mode regressions. - Server and UI TypeScript checks, SDK build, verification-map validation, credential-pattern scan, added-comment scan, and `git diff --check` pass. - A downstream integration uses this Core resolver and preparation service with a real storage hook and mocked remote infrastructure. A new Git project reaches the hook without a stored provider ID. - Standard Greptile and Apex both scored 5/5 on 5d42998466. All review threads are resolved. - Final GitHub CI is fully green on 5d42998466. Browser shard 3 initially hit a PostgreSQL deadlock during cleanup cancellation, which left an experiment flag enabled for a later test. A failed-job retry passed all 26 browser tests. The workflow completed successfully on attempt 2. Full final verification was delegated to CI at the user's request. - No live provider or hosted tenant run was performed. ## Risks - Installing one default-claiming provider changes effective selection for unconfigured projects. Existing explicit settings and workspace ownership remain intact; incompatible explicit settings fail clearly. - Each configuration/admission resolution reads installed plugin metadata and the selected company's enablement. Project list reads share one catalog query. Selection does not call workers or external providers. - Deploy the Core contract before any downstream manifest uses the declaration. No migration or backfill is required. A downstream immutable image pin must include this change before packaging its declaration. - No merge or deployment is authorized by this PR. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run targeted tests locally and they pass; full gates are delegated to CI at the user's request - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on 5d42998466ad42400564ba34f48124f40c53fbc5; the branch has no merge conflicts - [x] Standard Greptile is 5/5 on 5d4299846 after all fixes, with no unresolved threads - [x] Apex is 5/5 on 5d42998466, after standard Greptile 5/5 on that same head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 17:12:29 -07:00
DottaandPaperclip	9bf7dc0c8f	feat(agents): bulk edit connections and default models (#17215) ... Add harness-scoped bulk connection and default-model edits to the agent roster, with compatible choices, change previews, per-agent save results, and retries. Use an icon-based harness filter, keep agent rows readable, remove the status selector and selection hint, and redirect legacy status URLs to the full roster. Preserve authorization, audit history, unrelated settings, and optimistic concurrency checks through the existing agent update route. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-22 18:47:06 -05:00
DottaandPaperclip	0a5f1546ee	feat(cases): add reusable prompt workflows (#17114) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Cases keep facts, documents, and one primary task across agent handoffs. > - Each case currently needs its own procedure text. > - Agents need to reuse procedures and move cases between them without replacing the task. > - This change adds company workflow definitions with a name and Markdown prompt. > - Existing handoff and document transactions preserve authority, revisions, and recovery. > - Cases remain behind `experimental.enableCases`. ## Linked Issues or Issue Description Refs #16959. Related proposal: #3951 uses task graphs and a visual editor. This change extends existing Cases with prompt definitions and keeps its current handoff model. **Problem or motivation** Agents cannot share a reusable procedure across cases or inspect a case's movement between procedures. **Proposed solution** Add name-and-prompt workflow definitions. Adopt a prompt into the primary task's revisioned workflow document through a handoff. Keep phases as free text. **Alternatives considered** Defer graphs, workflow versions, run entities, role schemas, authoring screens, and Spec Kit integration. **Roadmap alignment** This extends the existing Cases experiment and same-task review flow. It adds no work queue or new execution engine. The maintainer approved this scope in the implementation plan. ## What Changed - Add company-scoped definition storage, shared request schemas, REST routes, and four optional Cases tools. - Accept a workflow ID at kickoff. Agent handoffs can adopt, refresh, preserve, or detach a workflow. - Capture prompt text and the document base revision at acceptance. Apply workflow, phase, and document revision together after outgoing execution settles. - Hold conflicting document edits for recovery. Preserve cancellation, replay, and restart behavior. - Show current workflow names and captured transition names/phases in Work, history, and Audit. - Update instructions, API/tool contracts, capability maps, and the isolated workflow journey. - Pin bounded workflow prompt reads to a content fingerprint; continuation conflicts on edits and can restart safely. - Remove experimental Cases/workflow/Mermaid guidance from the always-on native prompt. Extract runtime-context prompt strings into a CODEOWNERS-protected constants file while preserving the public imports. - Refresh Copilot’s pinned source identity to v57, including the extracted prompt file and retaining historical identities. ## Verification Local checks: repository `pnpm -r typecheck` and `pnpm build` pass on the prompt-extraction tree (`88222d45d8`). Full repository coverage previously completed through the supported isolated shard groups (1,082 general server suites, 161 serialized suites, and all three workspace groups). The earlier default serial run was interrupted and remains incomplete. The first sharded run retained the failures described below; targeted corrections and separate limited-concurrency reruns pass. All CI jobs, security checks, and the 5/5 Greptile review passed on `fc1bd79c28`. Fresh CI and review are pending for the prompt extraction in `88222d45d8`. Focused verification passes: real-database handoff recovery and reuse; 2,002 native-tool/API contract checks; 144 Runner catalog/discovery checks; 27 Runner context/coverage checks; 37 case/Audit UI checks; and Product E2E oracle calibration. Token gates, migration safety/numbering, module boundaries, capability maps, and generated tool/Storybook catalogs pass. The final route/context corrections pass 49 API/native-tool tests, 90 real-database handoff tests, and server typecheck; 142 Copilot installation/sidecar tests pass with the refreshed identity. Greptile’s first review found that successive prompt pages could mix definitions after an edit. The fix returns a content fingerprint, requires it for continuation, and rejects edits with 409. Strict live/catalog schemas pass (171 Runner checks). The 2,005 API/OpenAPI tests pass; the initial native regression caught compression’s weak ETag prefix, and all 23 native integration tests pass after handling it. The regression covers same-length edits and a four-byte restart window. No stored version fields or entities were added. The prompt extraction passes all 462 checks across six focused suites, including persisted-context recovery and provider transport. The first local transport attempt lacked the required debug Runner binaries (102 failures, retained); after building the normal prerequisites, all 283 tests in that suite pass without source or assertion changes. Existing saved snapshots retain their recorded prompt text, and new executions use core-only prompt v7. The extracted prompt source is now covered by both CODEOWNERS and Copilot’s source fingerprint. Live-provider evidence is separate from those simulated checks: the isolated API-key campaign `reusable-workflows-api-final-2026-10-21` passed with local code matching `a1d73b4652`. Four actual Codex API runs read the adopted task documents, moved one case A → B → A on the same task, completed normally, and passed all nine durable checks. Browser assertions verified the current workflow/phase, historical names/phases, five handoff records including terminal completion, and the completed original task. Cleanup passed. The local report remains under `tests/runner-e2e/results/reusable-workflows-api-final-2026-10-21/`; no credentials or screenshots are committed. CI initially caught an omitted route-inventory entry and a stale pinned instruction-source hash; both are corrected and separately verified. Local serialized coverage also recorded an unrelated environment-route socket disconnect and activity-route timeout; their separate 11-test and 17-test suites passed without code or timeout changes. An unchanged chunked-webhook check also passed its separate four-test rerun. The initial adapter/database workspace groups hit time and resource limits under default worker counts; complete reruns with `VITEST_MAX_WORKERS=2` pass, including all 55 database files and 211 tests. The UI/CLI groups passed 837 files and 9,442 tests in the original run. Original failures remain retained and are not regraded. Earlier failed live campaigns remain unchanged: subscription HTTP 429 before agent execution; an obsolete exact-four-row oracle that rejected the valid terminal fifth row; and a chat-marker expectation that did not account for the existing explicit-comment presentation policy. The final fixture requires terminal completion and places the marker in its authoritative comment. Positive/negative oracle tests and the fresh live campaign pass. ## Risks - The additive migration adds a table and nullable case/handoff fields. Existing rows keep null workflow metadata; no assignments are inferred. - Adoption can wait on an intervening task-document edit. Recovery must cancel the old intent and accept a new one after readback. - Shared definition edits affect future adoptions. Existing case copies keep their revisions. - The feature remains experimental. This PR does not merge or deploy the change. - Live acceptance covers native Codex before the prompt extraction; this cleanup has contract/transport verification rather than a new live-provider campaign. The refreshed Copilot identity has static compatibility evidence, not new live-provider qualification. ## Model Used OpenAI Codex, configured model ID `gpt-6-astra`, xhigh reasoning. Used terminal execution, file editing, and tool calls. The active context-window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 15:00:12 -05:00
DottaandPaperclip	3ce8d976a9	Add experimental Paperclip Notebook MCP connection (#16984) ... ## Thinking Path > - Paperclip connects agents to external tools through saved accounts and per-action permissions. > - Notebook exposes artifact management through an ordinary remote MCP server with OAuth. > - A private pilot needs a hidden, default-off catalog and execution gate. > - Local file publishing belongs to Notebook, which can accept bounded file bytes through ordinary MCP tools. > - Paperclip therefore needs the connector registration and generic availability checks, with no file-transfer architecture. ## Linked Issues or Issue Description **Subsystem affected** Apps catalog, experimental feature metadata, connection setup, and connector availability. **Problem or motivation** Users need to connect Notebook as their Cloud account, assign its tools to agents, and retain Paperclip's normal Allowed / Ask first / Off rules. The private pilot must remain hidden until an operator enables it. Notebook should own its upload protocol and local publishing script. **Proposed solution** Register a personal remote MCP OAuth connection at Notebook's canonical endpoint. Reuse existing OAuth discovery, credential storage, refresh, reconnect, and removal. Enforce availability through generic catalog feature metadata and saved application identity. Keep Notebook's standalone uploader in its own service repository; it calls the ordinary gateway and receives no saved OAuth credential. **Alternatives considered** A gallery-only filter permits retained tools and deep links while disabled. A shared workspace publisher and private upload-ticket transport add runtime behavior that a normal MCP connector does not require. Notebook can receive file bytes directly through its own tools instead. **Roadmap alignment** This extends Connected Apps and supports Artifacts and Work Products. ROADMAP.md and related PRs were checked. This PR is standalone on master. The separate OAuth lifecycle and file-transfer proposals (#17085 and #17086) are closed and are not dependencies. ## What Changed - Register Notebook's personal MCP OAuth method, canonical endpoint, scopes, catalog metadata, and branding. - Declare managed `enablePaperclipNotebook` with false defaults and no user-facing settings toggle or staging opt-in. - Use generic availability metadata for gallery/setup, retained connection identity, OAuth completion, health, tool/context discovery, and dispatch. Resources/prompts recheck before credentials, including retained handles. Removal remains available while disabled. - Resolve catalog application identity from the saved application key, so editable connection labels cannot bypass the hidden gate. - Select a method's sole supported credential kind when no kind is supplied. - Keep UI changes to metadata-driven availability and existing setup/reconnect controls. - Document Notebook's service-owned upload script, inline publication, ordinary action policies, and credential behavior. - Add ordinary MCP OAuth, assignment, action-policy, disabled-state, settings, and catalog tests. No workspace contribution registry, private transfer endpoint, native publisher, Runner file-reader change, or Paperclip CLI upload command is included. The entire native-runtime and CLI trees match master. Upload code lives in `paperclip-cloud/notebook/`. ## Verification Source head: `65ef741e4f476c67425b57d2fb2d6d4c00e99b06`, directly on master `53732c811bb69655eb43d3d12f2063ae4ca5ad67`. The diff is 42 files. Shared execution changes only enforce generic catalog availability; tool access also resolves saved application identity and the sole supported credential kind. Neither module contains Notebook upload logic. - Catalog/shared/settings/UI suites: 592 tests passed across 12 files. Shared/server/UI typechecks passed using installed compilers. - Fresh real PostgreSQL generic MCP suite: 84/84 passed at final source; server typecheck and module boundaries pass. Coverage includes personal OAuth, ordinary refresh/reconnect, assignments, Allowed / Ask first / Off, hidden feature enforcement, and local credential removal. Four new resources/prompts cases failed against the prior code and pass after filtering saved application identity and rechecking before credentials. They prove missing/relabelled setup markers and stale eligibility cannot dispatch or read secrets while disabled; an ordinary provider remains available with the flag off. - The full tool-access service suite passed 463/464 cases. One unchanged Google Workspace `slides.write` case timed out at 15 seconds; it passed on one isolated retry without source or timeout changes. Its exact test block matches master. The cause is not established; final-head CI must qualify the complete suite. - Maps, module boundaries, token gates, and whitespace checks passed. Byte comparisons confirm the former transfer/runtime/CLI surfaces match master. - Fresh exact-head Greptile is 5/5 with zero unresolved findings after an exhaustive feedback audit. Current-head CI workflow [39625691137](https://github.com/paperclipai/paperclip/actions/runs/39625691137) succeeded, including both required checks (`ci / verify`, `ci / e2e`), Build, and browser E2E. Supplementary security/advisory scan status is recorded in the PR Checks panel and qualification receipt; prior-head scans are not counted. The superseded `4bd7af9b2e` completed CI successfully, but Greptile found the valid resources/prompts availability gap described above. It was fixed, replied to, and resolved; earlier-head results do not qualify this revision. Preserved environment failures: the first socket-restricted UI/route run had 93 listener setup failures, and the same 93 cases passed with local-socket access. The package-manager launcher could not fetch its signature metadata; installed compilers passed without dependency or configuration changes. The full service timeout and isolated retry remain recorded separately. The superseded `585e870e73` CI also failed an unchanged agent-chat project-plan revision assertion after task creation; the complete spec and relevant runtime sources match master, and a same-master prior run passed. Its cause is not established. Logs are preserved; no test or timeout was weakened and no old-head rerun was requested. The subsequent heads run full CI independently; no prior-head pass substitutes for current qualification. The Notebook uploader requires a separate service release. This public PR adds no uploader runtime to Paperclip and claims no newly deployed browser/agent acceptance. The Notebook-owned upload path is qualified separately. Code-owner review remains required before merge. ## Risks - The feature flag gates the curated connector; it does not create a Notebook service-side team allowlist or block arbitrary standard MCP URLs. - Publishing, sharing, and deletion retain the existing assignment and action-policy boundaries. - The hidden feature must be checked for retained tools and saved identity, including when connection labels change. - Disconnect removes local credentials. It does not revoke the remote Notebook authorization; the owner can revoke it in Notebook. - Ordinary refresh uses the existing client. A lost response after refresh-token rotation can require reconnect. - The standalone Notebook script and its bounded upload tools must ship together in the Notebook service before that documented convenience path is available. - No database schema, workflow, lockfile, shared transfer, or Runner change is included. No merge, deployment, or rollout is performed by this update. ## Model Used OpenAI GPT-6 in Codex, with reasoning, repository inspection, code execution, and tool use. The runtime did not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-21 11:07:18 -05:00
DottaandPaperclip	d33b40a5c3	fix: recover saved work and provide reliable task Retry (#16965) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The control plane must retain work when an execution or workspace export fails. > - Retained-workspace holds can leave saved input waiting with no usable Retry action. > - Users cannot verify process ownership or repair provider allocations. > - This pull request gives system recovery a durable budget and gives each deliberate Retry a durable intent. > - Safe recovery can preserve files, settle an accepted result, or prepare an independent workspace. > - One task state keeps infrastructure diagnostics out of the ordinary conversation while preserving real permission and approval gates. ## Linked Issues or Issue Description **What happened?** A task with a retained workspace can show recovery, failed-run and saved-message warnings at the same time. Historical runs without native workspace descriptors cannot use the native restore path. Old recovery metadata can prevent a new attempt even when an independent workspace would be safe. **Expected behavior** Paperclip owns bounded infrastructure retries and file preservation. A user can request another attempt with ordinary Retry. Thinking appears immediately. The service preserves completed effects and current user controls. It does not claim that starting fresh recovered the old files. **Steps to reproduce** 1. Retain a failed run and its workspace after export or process-stop failure. 2. Save a follow-up while the recovery hold is active. 3. Open the task and try to continue without infrastructure credentials or operator certification. Refs #16906, #16835, #16883. This extends their recovery and receipt contracts. Related work: #16962 and #16963. ## What Changed - Add task-level Retry with company-scoped authorization, all-status idempotency and durable admission before physical work. Keep exact-run Retry compatible. - Add typed recovery phases, finite persisted backoff, historical adoption and durable local failure diagnostics. - Preserve native workspaces or verified historical snapshots. Admit an independent fresh workspace only after execution containment. Keep old-source preservation and cleanup obligations separate. - Carry completed publication and API receipts forward. Fence uncertain mutation retries across replacement runs. Preserve connected-chat identity and authorization. - Re-enter normal dispatch for saved input. Reuse accepted results through the existing finalizer. - Use one task interruption projection across the thread, composer and task list. Show optimistic Thinking, ordinary Retry and genuine user gates. - Add PostgreSQL, filesystem, API, browser-fixture and controlled provider coverage. Document the behavior and update the feature/access maps. ## Verification - All GitHub Actions CI jobs pass on `6d8c6d27d50586d8548b31d03b47f6d73a0ee11d`: typecheck, all server/chat/workspace and serialized suites, Runner checks, eight browser shards, canary and build. The required merge gate passes. - Greptile reviewed the final commit at 5/5, with no actionable findings or unresolved review threads. Its two earlier P1 findings are fixed. - Full recursive typecheck and build pass locally on the final commit. All 16 task Retry cases pass against isolated real PostgreSQL. The unrelated-success case and both stale Retry cases fail on the previous code and pass after the fix. - The corrected gateway fixture passes its uncertain-mutation and independent-read case. Four real HTTP API cases pass for definite 403/422 rejection, completed-effect reuse, and uncertain-effect fences across deliberate and automatic retries. These focused gateway/API checks used a temporary database lifecycle harness on an existing healthy local PostgreSQL process. The standard repository helper is unchanged. - Earlier qualification covers 291 task UI cases, saved-file copyback and ancestry, current Agent Chat ownership races, nine chat Retry API browser cases, and six assembled browser recovery fixtures. A real API/PostgreSQL/native-runner journey showed Thinking in 25 ms despite a delayed response, admitted one successor in a separate workspace, retained original files, and survived reload. - The controlled Daytona test interrupts export, joins the controller, restarts it in a separate process, and verifies one agent start, one external sentinel operation, retained file contents and no premature cleanup. It does not establish abrupt-kill or live LLM/GitHub acceptance. - The full local `pnpm test:run` was attempted and was not green: an OAuth row-lock timing assertion failed, and a later suite exhausted macOS PostgreSQL semaphores. The ten OAuth timing cases passed on rerun against a separate database. Later embedded database initialization failures remain recorded; the final CI matrix passes. - Security scanning passed on production-code commit `63953edf069b111c6becc3144dffa6deb52f68b5`. Its only successor changes one test-fixture identity line. The final scan is neutral/inconclusive because its pinned merge-base checkout failed or timed out. Both rerun API requests returned 404, so no fresh successful scan is claimed. Socket and Snyk checks pass on the final head. - The optional Paperclip Review remains queued behind an earlier run in workspace finalization. Its prior completed assessment was incomplete because a comparison tool was blocked by the platform filter. It is not counted as a completed code review. GitHub does not require this check for merge. - Hosted acceptance remains pending deployment. Local and CI results do not establish recovery of a hosted task or its retained files. ## Risks - Recovery changes span admission, physical ownership and dispatch. Incorrect evidence could permit overlapping work. Existing physical locks and authoritative stop requirements remain in place. - The migration adds concurrent indexes. It requires the existing top-level migration runner and retries incomplete index creation safely. - A fresh attempt can complete while old-source preservation remains unresolved. The service retains that obligation and does not report those files as recovered. - Unknown tool mutations remain blocked until their authoritative receipt owner resolves them. Generic upstream tools have no universal result-query protocol. - Recovery and diagnostics stay inside the stack. No Cloud incident endpoint, notification worker or new outbound reporting channel is included. ## Model Used OpenAI Codex, GPT-6 family. The runtime does not expose a more specific model version or context-window size. The model used reasoning, shell execution, source editing and browser tools. No model subagents were used for this implementation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; the full local run limitation is documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 19:58:53 -05:00
Nicky LeachandPaperclip	8b4ab65b1c	Add plugin-owned project storage and task workspaces (#17001) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description **Subsystem affected** Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. **Problem or motivation** Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. **Proposed solution** Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. **Alternatives considered** Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. **Roadmap alignment** This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 16:35:33 -07:00
DottaandPaperclip	505e1a5582	feat(cases): add persistent workflows with same-task agent handoffs (#16959) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - An agent workflow needs persistent facts and a safe way to transfer work. > - Creating a task for each phase splits the objective, discussion, workspace, and review policy. > - A case can retain the facts while one primary task moves between agents. > - Transfers must survive process failure and preserve private access, budgets, and completion gates. > - This pull request adds durable case handoffs, native case tools, and read-only case pages. > - Prompts describe the workflow. Existing task controls decide when work can run and finish. ## Linked Issues or Issue Description **Problem or motivation** Prompt-driven agent teams need a persistent record that can move through triage, implementation, verification, and correction. Ordinary reassignment does not provide a durable instruction, safe release barrier, or recoverable delivery receipt for an agent handing off its own running task. **Proposed solution** Extend standalone cases with one primary task, a descriptive phase, retained access sources, and a handoff ledger. Apply assignments after source execution and workspace settlement. Deliver each transfer through the existing durable wake queue. Provide `create_case`, `read_case`, `update_case`, and `handoff_case` in the native runner. Creation requires kickoff and explicit workflow text. An explicit handoff to `done` closes the case when normal task completion is accepted. **Alternatives considered** Do not add a workflow engine, filesystem queue, configured state graph, or one task per phase. The workflow document combines transition prose with an optional Mermaid `stateDiagram-v2` map. The server does not execute the diagram. The Pipelines experiment removed on master stays removed. **Roadmap alignment** This extends Agent Reviews and Approvals, Enforced Outcomes, Bounded Run Recovery, and Activity Log attribution. Existing approval, review, budget, privacy, workspace, and responsible-user rules remain authoritative. Related work: [ordinary handoff wake recovery](https://github.com/paperclipai/paperclip/pull/15019). This feature adds a separate persisted case-handoff receipt and release barrier. ## What Changed - Add case routing fields, retained access sources, durable handoff history, delivery leases, and replay-safe migrations. - Keep the same primary task across agent phases. Preserve each returning agent's own session and the task's workspace. - Fence outgoing mutations, wait for provider and workspace release, and recover delivery without duplicate task comments or runs. Recheck current assignment permissions as the original caller before applying a delayed transfer. Scope a run Stop to that run’s own handoff intents, so repeated old Stop requests preserve newer work. Cancelling terminal intent without a source run also requires fenced current task ownership, including native retries and preparation with an old checkout. For Cloud callers, revalidate current stack membership with an uncached Cloud RPC and derive owner elevation from the current managed setting. - Keep case reads, documents, assets, Audit, and live events inside retained task/project permissions. - Add native case creation/read/update/handoff tools. Store required workflow text and initial documents on the first task. Page oversized case facts and referenced document bodies through the dedicated tool with bounded UTF-8 JSON, current routing versions, snapshot conflicts, and current source authorization on every page. - Add read-only case lists and detail pages, task-style list controls, compact dotted field tables, linked handoff tasks, and case filters in Audit. - Add native/legacy integration tests, migration/restart tests, browser coverage, and bounded live receipt fixtures. Extend the trusted sandbox condition to cover the new subscription fixture profile. - Hide obsolete paused-agent warnings on completed or cancelled task threads. - Synchronize the company-switch settings test with React effects after a CI failure. Preserve its security assertions and production remount behavior. ## Verification The branch directly includes master `9ef434fe79`. The required `ci / verify` and `ci / e2e` gates pass on head `765fccd67c`. [Current-head CI](https://github.com/paperclipai/paperclip/actions/runs/39464348614) passes typecheck, build, tests, and all eight browser shards. Security checks pass. [Greptile gives this exact head 5/5](https://github.com/paperclipai/paperclip/pull/16959#issuecomment-6291156549), with no unresolved review threads. The optional Paperclip Review check remains queued because its assigned agent is paused. It is not a required branch check. No check result was overridden. Local workspace typecheck and build pass after the master merge, along with database migration checks, token gates, map checks, Storybook catalog checks, and the Copilot profile identity check. Standard builds refresh Runner vendor output. Focused checks pass 2,008 API/paging/context tests and 7 strict case protocol tests. The final commit changes one test fixture only; its ten tests, UI typecheck, and token gates pass. No production source changed in that commit. On current head `765fccd67c`, all **194 critical workflow tests** executed and passed with no skipped tests in those files: - 38 Stop/ownership tests, including board completion intent, native retries, stale Stop, and preparation with an old checkout. - 2 actual legacy process/HTTP handoff tests, including scratch failure, a closed-stdio child that continues writing, and recovery through a fresh worker after the process group stops. - 5 migration tests, including SQL column metadata, preserved null authority context, and all six prior-numbering receipts. - 3 process-settlement and 5 scratch tests. - 88 handoff-service, 18 heartbeat handoff, and 21 native case-tool integration tests. - 7 bounded case/document-read projection tests and 7 strict protocol-contract tests. The current CI checkout was `fed65d49`, merging `765fccd67c` into master `9ef434fe79`. Its new document regressions reconstruct exact Unicode/escaped bodies and revisions through real native authority and authenticated HTTP with generic API tools disabled. They cover case-owned, primary-task, and linked-task documents; routing on every page; stale document snapshots; access loss and unlinking; unrelated/foreign/private sources; system documents; and quarantined content. Active-task document tools retain their existing behavior. These are current-head test results from the first CI attempt; the browser-only retry did not rerun these suites. The full `64fdd3403f` CI failed one separate settings test: after a company switch, it observed an unsettled API-key draft. The production component and test were unchanged from prior green source. Production switches remount both keyed settings layers; the test directly reused an unkeyed view and used a timer tick instead of waiting for React effects. The final commit replaces that fixture synchronization with React `act`. The exact secret-clearing assertions and all timeouts remain unchanged. The original assertion failure remains recorded; no unchanged retry was used. The narrow unchanged baseline and fixed fixture each pass 10 tests. The upstream result-paging merge preserves case capture namespaces and current asset privacy. Its API integration suite executed 39 of 41 tests successfully, with two skipped; the optional assigned-result PRP integration test was skipped. Neither skipped suite is counted in the 194 critical tests or as live provider proof. All 361 upstream migration entries and SQL bodies remain unchanged. Six case migrations follow master as 0363–0368 with identical SQL hashes. Official Drizzle generators produced the combined metadata and five retained snapshots. Generation reports no schema changes, and the snapshot drift test passes. The earlier `ce361ae5d0` CI passed on attempt 1, including its then-current 179 critical tests. Fresh review later identified unusable document references with generic API tools disabled; the bounded document reads and ten additional authenticated HTTP tests address that finding. The full local `pnpm test:run` attempt was cancelled after this Mac exhausted PostgreSQL semaphore capacity; it is **not** counted as a pass. Later database attempts also stopped before assertions or skipped, including one sandbox socket denial. Two original stale-Stop regressions did reproduce the bug against the old source. A REST serialization assertion passed after two preserved database-startup failures. No test timeout was increased. All 194 assertions listed above executed and passed in current-head CI. The complete 385-file, 3,963-test UI shard also passed after the fixture repair. A configured-provider receipt workflow passed 13 backend checks across triage → implementation → verification → correction → done, using one task and four work runs. The saved receipt is revision 2 with total 6 USD; case and task both closed. The original canonical attempt remains failed because its browser assertion confused the terminal destination with completed-task tooltips. The corrected full browser verifier passed on the same retained database and original runs, with no additional provider calls. The last clean-build browser readback is on `6a86c54a53`; later changes affect recovery, authorization, bounded reads, and settlement. Accounted work cost is a $1.010574 rate-card estimate; provider billing and setup-probe usage are unavailable. Copilot profile 49 passed the unchanged seven-fixture private protocol pack (46/46 checks, no retries) and the product file-edit canary after master's completion-replay change. All 60 bound implementation hashes remain unchanged on the current source. Historical profile-47/48 identities and results remain intact. See the [profile-49 compatibility record](https://github.com/paperclipai/paperclip/blob/codex/persistent-case-handoffs/doc/plans/2026-10-20-copilot-completion-recovery-compatibility-v49.json). This is bounded compatibility evidence, not a fresh full matrix, installed-package product test, or Daytona rollout. Preserved CI failures and retries: - On `20662930ac`, Slack managed setup failed a five-second success assertion after reload. Its test/application path was unchanged from a prior green head. One unchanged-source retry passed all 25 tests in that shard; no source or timeout changed. - On `9402b4f17a`, two CI workers received shutdown signals at 15:24:19 UTC. Neither log had an assertion failure before interruption. One unchanged-source retry of each passed all Runner static checks and all 32 tests in browser shard 6; both aggregates passed. The next head, `7b4374edaf`, passed all CI without retries. - On `295f289b5a`, serialized shard 9 timed out during its first device-login test, then the next test found an active login lease. Direct login/test paths were unchanged from a prior green head. Cold import plus a timed-out async body using the next test's mutable actor/store is the supported fixture-race explanation; no request trace proves that sequence. The original failure remains recorded. One unchanged-source retry passed all 294 tests in 19 files, including all 41 device-login tests. No login source or timeout changed. The subsequent `ce361ae5d0` CI passed without retries. - On `64fdd3403f`, one company-switch settings assertion observed an unsettled draft. The final test-only commit uses React `act` without changing assertions, timeouts, or production code. On `765fccd67c`, all ten settings tests and the complete 385-file, 3,963-test UI shard passed. - On `765fccd67c`, browser shard 3 completed the split-handoff assertions, then cleanup received HTTP 500 from run cancellation after a one-second PostgreSQL row-lock timeout. The extra chat run was reporting completed work while teardown began. Cleanup stopped before restoring a global experiment flag, causing a later assertion to fail. Relevant production and test paths are unchanged from an earlier green commit. Stop's lock order is consistent, and no nested self-lock was found; the exact PostgreSQL blocker was not recorded. One unchanged-shard retry passed. The original failure, trace, and downstream flag failure remain recorded. No Stop implementation, cleanup assertion, or timeout changed. The retry supports intermittent cleanup contention; it does not identify the original blocking transaction. ## Risks - Local process transfers require host-owned PID/group absence. Missing or uncertain termination evidence keeps delivery pending, independently of scratch setup. - Handoff admission and task cancellation share execution ownership. New regression tests cover temporary holds, late source writes, and uncertain provider termination. - Retained private sources use an intersection of existing permissions. This intentionally fails closed when a source is missing or an existing task cannot preserve that boundary. - Migrations add tables and indexes. They preserve master history and can replay over prior feature installs. Migration 0368 adds nullable internal authorization provenance with no default or authority backfill. Historical transfers still awaiting release without this context fail closed; an operator must cancel them and request a new handoff. - Delayed Cloud handoffs depend on the current Cloud membership RPC. An unavailable, ambiguous, expired, or mismatched response holds the handoff for retry; it does not reuse cached authority or grant access. - Bounded reads reload and reauthorize the selected projection per page. Underlying work grows with the data size and number of pages. Changed data or routing requires restarting from offset zero without the old snapshot. - Phase names and workflow diagrams remain advisory. They do not satisfy review or approval gates. - This is a large, cross-cutting PR. The maintainer requested one PR for the whole feature; the commits separate data, runtime, UI, and qualification work. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and delegated code review. The exact deployment model ID and context-window size were not exposed in the session. Live fixture agents used the explicitly configured provider model recorded in their test evidence. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 12:46:14 -05:00
DottaandPaperclip	2c96f76780	feat(connections): add model-backed prompt injection prevention ... Add opt-in OpenRouter checks at existing tool-result boundaries with regex fallback, service accounting, and settings on Connections. Make migration 0362 safe to replay over existing or partially applied schema state. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-20 11:32:25 -05:00
DottaandPaperclip	bc091f0a21	fix(recovery): recover retained workspaces before saved messages (#16906) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A stopped run can leave useful files in its original sandbox. > - The server keeps that sandbox when it cannot confirm file copyback. > - The task can then save new user input but offers no safe way to recover the files. > - A retry of the old agent turn can repeat work that already finished. > - This pull request recovers files through the existing workspace owner and delivers only saved human input. > - The user can recover a task without certifying infrastructure facts or losing completed work. ## Linked Issues or Issue Description Refs #16883 and #16835 for publication receipts and verified process termination. Related: #16478 clears stale holds after copyback already succeeded. This change performs the missing export and records its proof. #16901 manages the main GitHub response comment and does not perform workspace repair. **What happened?** A failed run retained its workspace. A later user message stayed saved behind a file-repair hold. The task asked the user to verify process termination and file recovery but offered no repair control. Sending a message could show an optimistic Working state and move the task to Todo before admission succeeded. **Expected behavior** The harness should recover the original files when it can prove ownership and termination. It should preserve newer edits and completed operations. Only waiting user input may start another turn. Missing evidence must keep the source held with a specific explanation. **Steps to reproduce** 1. Interrupt a sandbox run before workspace finalization succeeds. 2. Retain its exact allocation and save a user follow-up. 3. Observe the file-repair hold and missing end-user repair path. ## What Changed - Add source-bound background recovery for native and legacy runs. An accepted native result is not required. - Share the physical copyback lock and durable owner. Resume only the exact stopped allocation, export files, verify persistence and obtain a fresh stop receipt. - Persist retry limits and idempotent requests on the existing recovery record. Limit each server to two background exports. Add an indexed candidate scan and a separate cleanup obligation. - Add a task-editor recovery endpoint. Deny agents and viewers. Reject caller-supplied stop or repair claims. - Deliver waiting human input once through ordinary admission. Preserve pause, approval, budget and assignment gates. Require historical conversational-adapter evidence for legacy-to-Runner continuation. - Include exact GitHub publication receipts in continuation context. Preserve unknown effects and the original failed run. - Show observed recovery progress, a transient-failure retry action and specific service blockers. Keep saved messages visible. Suppress false startup feedback and implicit Todo promotion behind a file hold. Hide unavailable workspace links. - Update execution semantics, capability maps and production-component stories. ## Verification - Red: the original read model failed the new recovery-progress assertion; the other 23 checks in that file passed. - Passed: recovery regressions with 28 cases with real PostgreSQL, files, tar and Git operations. Lifecycle provider calls are controlled test boundaries. Coverage includes native/legacy export, concurrent owners, rollback after export, interrupted archive export with a new database connection, lost replies, historical discovery, durable retry limits, separate cleanup and saved-input delivery. The actual candidate query uses its partial index with 3,000 unrelated history rows. A first automatic repair can bind verified old-controller stop proof without an accepted native result. Review also reproduced and fixed export-only continuation and cleanup lost after a committed repair. Cleanup remains discoverable without waiting input and after task reconciliation. - Passed: two GitHub integration cases verify that continuation context carries the existing publication receipt and does not write another response. Company and task boundaries are checked. - Passed: 293 continuation, workspace-sync, platform-receipt and OpenAPI checks. After file repair, actual dispatch retains pause, cancellation, approval, real budget-policy and assignment gates. - Passed: 174 route checks, including editor admission, agent/viewer denial and rejection of fabricated repair claims. - Passed: 351 task detail, workspace properties and comment-reopen checks after updating the new read boundaries and asynchronous workspace-link expectations. Two new route cases keep held tasks Blocked when a comment is saved. - Passed: 265 task-thread and recovery-notice checks. The broader continuation suite also passed; its new legacy migration case first exposed a gap that the focused recovery suite now covers. - Passed after rebase: `pnpm -r typecheck` and `pnpm build`. The later master merge adds an unrelated permission migration; recovery migration generation and safety checks passed after the merge. - Passed: token gates, module boundaries, map structure and migration checks. The recovery index is migration 0360 after current master. - Browser: inspected the production recovery component and its retry-to-queued transition with fixture data. The first dev dependency load failed; a reload after optimization resolved it. This is UI evidence, not a live provider recovery claim. - Every test shard passed on final commit `bbbc09efb4`. The older local `pnpm test:run` was stopped after the complete CI test matrix passed. Local full typecheck and production build passed. The first CI build compiled and booted, then lost its runner during the clean public npm-install check. An isolated retry passed the build, production boot and clean public npm-install verification. - Greptile reviewed final commit `bbbc09efb4` at 5/5. All review threads are resolved. - The change is not deployed. Hosted recovery remains unverified until the served revision includes this change. ## Risks - Recovery crosses process, provider and persistence boundaries. Missing or mismatched ownership, stop, descriptor or file evidence keeps the hold. It never selects a replacement allocation or silently starts over. - File recovery does not certify external action outcomes or authorize replay. New input still uses normal execution admission. - Historical sources without sufficient durable evidence can remain blocked. Destructive reconstruction stays separate. - The index migration is idempotent. It changes no task data. Snapshot pruning follows repository policy. ## Model Used OpenAI Codex, `gpt-6.1-sol`, with code execution and browser tools. `gpt-6-astra` assisted earlier investigation. The runtime does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 21:51:46 -05:00
DottaandPaperclip	ac9459bae3	feat: configure company agent observability (#16874) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Companies use several agent harnesses and execution environments. > - Operators need one place to select where native agent logs and traces go. > - An inference proxy would add a dependency to the agent request path. > - This pull request supplies native OpenTelemetry settings to managed Codex and Claude Code processes. > - Company secrets protect destination credentials. Export failure does not change the task result. ## Linked Issues or Issue Description **Subsystem affected** Company settings, shared contracts, database, server, adapters, and runner. **Problem or motivation** An operator cannot configure a company destination for native agent logs and traces. Manual harness configuration is hard to apply across local and remote execution. **Proposed solution** Add one company destination: Braintrust or a custom OTLP HTTP/protobuf endpoint. Apply a transient snapshot when execution preparation starts. Keep credentials out of API reads, task environment manifests, recovery files, and command arguments. Replace a warm process when its export settings change and preserve its conversation. **Alternatives considered** Multiple destinations can use a user-operated collector for fan-out. This change does not add a collector, an inference proxy, or transcript scraping. **Roadmap alignment** This extends company settings, secret management, and support for multiple harnesses. The maintainer requested this implementation. Searches for observability and Braintrust found no duplicate company destination feature. Related infrastructure tracing work in #16236 has a separate content-free contract. ## What Changed - Add strict schemas, revision checks, encrypted secret storage, activity records, and board-only settings routes. - Add explicit synthetic logs and traces tests with a bounded HTTP request and sanitized results. - Supply native Codex configuration and Claude Code environment values through local and remote launch paths. - Preserve Codex login refresh through the existing managed credential link. Retain unsaved drafts across background reads. Require complete custom header re-entry when the endpoint origin changes. - Separate process export compatibility from conversation identity. Isolate generated Codex configuration and preserve resumable sessions, including remote home refresh after export changes. - Advance the immutable Pi/Copilot source attestations for shared environment and sidecar code changes. Keep their provider payloads unchanged and retain prior identities for decoding. - Redact destination credentials throughout launch diagnostics and HTTP logs. - Add the production settings form, Storybook states and journey, documentation, and feature/access map entries. ## Verification - Real managed local Codex 0.160.0 and Claude Code 2.1.286 tasks exported to Braintrust. API readback and the Braintrust UI showed the test records. A real remote Daytona Codex task exported through the legacy `exec` path. - Live tests exercised content capture off, export disabled, warm conversation reuse, and a destination that accepted connections but never replied. Agent work succeeded and export requests stopped at the configured three-second timeout. The UI connection test returned “Test payload accepted”; the saved API key stayed masked after reload. - All current-head CI test lanes, typechecks, packaging/build checks, and required aggregates passed on `71e994b2be9c8fb6d6e938a4ad7cf4c95fdafe31` ([CI run](https://github.com/paperclipai/paperclip/actions/runs/39319194318)). Greptile completed on that exact commit at 5/5, with zero unresolved review threads. The additional optional Paperclip Review check remains queued waiting for its assigned agent. - Local workspace typecheck and build passed. Focused verification included 239 combined runner tests, 789 API/native-session tests, 107 Codex home/ACP/sandbox-site tests, 204 feature/privacy tests, and the affected Rust suites. Migration, token, map, Storybook catalog, and Storybook build checks passed during acceptance. These checks ran at the relevant implementation or merge boundary; the current-head CI is the complete final gate. - Managed AI credential projections now receive enabled and disabled company export settings; genuine caller-owned Codex homes remain untouched. Two regressions reproduced the skip before the repair. All 276 engine/lifecycle tests and adapter-utils typecheck pass afterward. The launched TOML retains provider/MCP settings, credential continuity, and redaction. - Settings saves, removal, and the real company-deletion owner acquire secret mutation ownership before database locks. PostgreSQL NOWAIT probes cover all three paths; concurrent save/rotation/removal/deletion requests settle correctly. The affected suites passed 137 tests, followed by all 32 service/connection cases after expanded deletion coverage. Server and Codex adapter typechecks pass. - Codex source profiles use the effective CLI argument list. A spawned fixture CLI reproduced both ignored-argument failures; all three precedence/fallback cases now pass. - Real tar staging reproduced remote conversation loss in both API-key and subscription modes before the repair. Both now pass: changing managed settings preserves sessions, archives, native indexes, and refreshed subscription credentials. A corrupt upload preserves the prior config and conversation. - Preserved failure evidence: initial CI caught stale Pi/Copilot source attestations, a missing test-harness helper, and packaging fixture/probe mismatches; those were corrected and covered by focused checks. Superseded heads had intermittent attachment-composer timeouts despite successful uploads; [the original trace and screenshot remain available](https://github.com/paperclipai/paperclip/actions/runs/39309701089/artifacts/12215109203). The same attachment case passed on the final head: [25/25 tests in shard 3](https://github.com/paperclipai/paperclip/actions/runs/39319194318/job/118288637051). - A connection-fault fixture also exposed a stale pooled socket after its hang case; closing fixture responses isolated scenarios and the full affected suite passed. - The broad local test run was not clean: database resource failures, timeouts, an older cached module graph, and a stale optional-resource manifest assumption were recorded. Focused checks passed after the documented fixes. This does not establish a clean full local pass. - Native remote Codex, remote Claude, and the full ACPX Codex bridge remain unqualified. The direct Codex resume test does not establish full bridge acceptance. ## Risks - Native exporters perform network requests from each execution environment. A server connection test does not prove remote reachability. - Prompts, assistant responses, and tool content are enabled by default when supported. Native caps and harness-specific controls still apply. - Caller-owned Codex homes are unsupported for automatic configuration and remain untouched. - Native harnesses and their tool processes can inspect export credentials. The settings form and documentation advise using ingestion-scoped credentials. Retained headers remain bound to their saved scheme, host, and port; changing origin requires fresh values or removal. - Migration 0357 adds a company-scoped row and secret reference. It is safe to reapply. Secret removal and company deletion use the existing secret lifecycle. - Slow or unavailable destinations must not block task settlement. Export settings take effect at the next execution preparation boundary. ## Model Used OpenAI Codex, GPT-6, with reasoning, tool use, code execution, and browser verification. The exact serving model ID and context window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 15:36:04 -05:00
DottaandPaperclip	3e5a9f5afe	perf(ui): load recent tasks with bounded summaries (#16851) ... Replace passive recent-task detail reads with bounded, authorized summaries. Keep account isolation, stored history, task actions, and live recovery behavior. Overlap access checks with shell settings and avoid duplicate session and health reads on the first healthy socket opening. Retain domain reconciliation. Add focused coverage and local performance evidence. Isolate the existing cache capacity test from entry expiry on slow CI runners. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-19 12:13:28 -05:00
DottaandPaperclip	9077748894	Remove the Pipelines experiment and preserve its data (#16834) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Pipelines was an unused experiment with its own UI, API, CLI, and automation. > - Keeping it leaves unsupported product surfaces and extra task-detail queries. > - Removing the experiment must preserve data that users already saved. > - This pull request removes the feature and keeps its database schema and migration history. > - Cases and Routines remain available as separate features. ## Linked Issues or Issue Description **What existing behavior does this improve?** Remove the unused Pipelines experiment across the application. **Current behavior** The experiment includes pipeline boards, stage settings, case automation, review queues, learnings, permissions, and CLI commands. Task detail also queries pipeline links. **Proposed behavior** Remove all these product surfaces and their implementation. Preserve existing tables, records, and documents. Ignore the retired boolean in older managed configurations so upgrades still start. **Breaking changes** Pipeline URLs, API endpoints, CLI commands, and `pipelines:write` are removed. Task detail no longer includes the old `linkedCases` field. The independent Cases API remains available. No database migration or cleanup is included. Related feature work: #7907, #9110, #10905, #11375, #13004, #14671, #16133. Related issues: #9663, #10310, #10312. This removal supersedes the feature; it does not implement those pending enhancements or delete their saved data. ## What Changed - Remove Pipeline UI pages, navigation, gates, clients, query keys, helpers, and unused design tokens. - Remove Pipeline REST routes, services, automation dispatch, task-detail queries, and OpenAPI discovery. - Remove the CLI commands, shared contracts, experimental flag, catalog entry, and permission defaults. - Remove feature-specific tests, tutorials, smoke scripts, maps, and Storybook examples. - Preserve database schema, migration history, and the reserved document key used by old records. - Keep Cases routes independent and cover obsolete stored and managed settings. ## Verification - Passed: `pnpm build` and `pnpm -r typecheck`. - Passed: targeted settings, managed configuration, sidebar, settings UI, routing, and feature-catalog tests. A first sidebar run caught a shared Workspaces icon import; the import was restored and the sidebar rerun passed. - Passed: 40 tests covering Cases routes, OpenAPI discovery, retained Pipeline table constraints, and schema snapshot drift. - Passed: token gates, map validation, generated API metadata check, Storybook catalog validation, and 28 map/catalog tests. - Passed: isolated local browser verification of navigation and Experimental settings with Pipelines absent; Cases and Routines still open normally. Live GET/POST requests to retired Pipeline endpoints return 404, and the retired setting is absent. CLI help has no Pipeline commands. - Passed: 22 access-service and agent-default-grant tests after restoring unrelated permission assertions flagged in review. - Local full run: `pnpm test:run` reported one failure in the unchanged workspace-runtime port-allocation test. That test passed in an isolated rerun. The long local run was stopped after full CI passed; it is not reported as a complete local pass. - Passed: all 56 checks on `def9181f3a4491d95fca1d38e9739f5b821aae3e` (2 conditional checks skipped), including build, typecheck, test shards, and browser suites. Eleven interrupted CI jobs reported “The operation was canceled”; their one-time retry passed. [CI run](https://github.com/paperclipai/paperclip/actions/runs/39255211006). - Greptile: 5/5 on the final commit, with the only review finding fixed and resolved. - Sandbox socket restrictions affected the first build/API test attempts. The same commands ran with local sockets available. ## Risks - Existing Pipeline clients must stop using the removed endpoints and commands. - Existing data remains in the database and backups, but the removed UI no longer displays it. - Historical boolean `enablePipelines` values are ignored. They cannot restore the feature. - Cases and Routines must retain their separate behavior. Targeted route tests cover this boundary. ## Model Used OpenAI Codex, GPT-6, with repository inspection, editing, shell execution, and browser verification. The exact deployed model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-19 08:09:46 -05:00
DottaandPaperclip	7c7d2d8d2c	fix(runtime): continue from saved sources after workspace loss (#16690) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runtime must preserve saved work and prevent duplicate execution. > - Older cleanup could destroy a sandbox before its workspace export finished. > - The existing recovery controls require either the original sandbox or recovered original files. > - Surviving attachments can support a rebuild, but they do not prove that missing edits were recovered. > - This pull request adds an explicit reconstruction decision and one fresh continuation through the normal execution gates. > - Operators can continue useful work while the original failed run remains truthful. ## Linked Issues or Issue Description **What happened?** A historical failed export can have an accepted result, a prepared workspace descriptor, and a confirmed destroyed sandbox. Its notice can also say `new_source_execution_path` and disappear. The task then has neither a truthful original-file repair path nor a supported reconstruction path. **Expected behavior** Keep the missing-file obligation visible. Let an authorized operator choose surviving task attachments and record lost work and external action outcomes. Start one fresh continuation after current execution checks. Keep the old result and failed export intact. **Steps to reproduce** 1. Seed a failed native export with a prepared descriptor and an exact destroyed-lease receipt. 2. Resolve its notice with the historical `new_source_execution_path` bookkeeping value. 3. Read the task. The baseline hides the hold even though no source reconciliation exists. 4. With this change, choose **Continue from saved sources** and submit a reconstruction decision. Related public work: #16451 preserves healthy warm sessions. #16583 prevents destructive cleanup of unexported work. This change handles historical source loss that those fixes cannot undo. Searches of open and closed work found no equivalent reconstruction endpoint. ## What Changed - Add a board-only reconstruction endpoint with `runtime:manage` and task visibility checks. - Validate the exact run, result, descriptor and destroyed lease. Reject active copyback, competing owners, newer work, stale task status, and foreign attachments. - Record attachment hashes, missing work, and external action outcomes. Preserve the original failed run, accepted result, prepared descriptor and terminal failure. - Hold other continuation paths until the durable reconstruction intent admits its successor. Record that successor in the same transaction and preserve the retry count. - Add an operator control that distinguishes reconstruction from original-file recovery. Update the recovery runbook and capability maps. ## Verification - Exact baseline source returns no blocker for the destroyed-source fixture. The same fixture returns a required recovery hold with this change. - New real PostgreSQL and assembled API/heartbeat suite: 25/25 pass. It covers duplicate submissions, lost acknowledgement, context reset, source identity, task ownership, concurrent execution attachment scope, later human blocks, deleted sources and agent pause. - Existing export retry, explicit continuation and recovery-action suites: 386 cases pass. One new-fixture constraint error in the combined run was corrected; the full new suite then passed. - UI recovery suites: 46/46 pass across three files. They cover disabled actions, loss acknowledgement, failure retention and runtime authority. - `pnpm -r typecheck` and `pnpm build` pass. The first sandboxed typecheck attempt could not open the TypeScript IPC socket; the authorized rerun passed. - Module boundaries, map checks, token gates and diff whitespace checks pass. - OpenAPI route coverage: 21/21 pass. The first CI head exposed a missing endpoint registration; this head adds the registration and the focused suite passes. - Review identified a missing dispatch-time check and an isolated attachment cache. Both are fixed. Claim and final handoff recheck the exact successor, sources and current execution gates. A controlled post-claim operator block prevents the assembled heartbeat from entering the provider. The shared attachment cache refreshes uploads and removes deleted selections. - Final focused run: 180/180 pass across six suites, including reconstruction, existing dispatch/stale-queue contracts and UI recovery. - A second review reproduced cancellation by an already satisfied historical connection card while the task was `todo`. Read-only connection availability now works before checkout; active-task mutation checks remain strict. The baseline failed two cases; after the fix all 74 connection and reconstruction tests pass, including revoked access, pending questions and pending approvals. - All current-head CI gates pass on `9d197236da11fa24f1ceadf5589cb2b81179d824`, including general/serialized tests, Runner, build and browser gates. Greptile is 5/5 for that exact head with no unresolved threads. The full local `pnpm test:run` is still running its serial server group; no full local pass is claimed. Live staging recovery is pending deployment. ## Risks - Reconstruction cannot restore missing bytes. The operator must identify surviving sources and actions that must not be repeated. The agent must verify those attachments before rebuilding. - This path is limited to exact, confirmed destroyed remote sources with an accepted result. Retained or uncertain sources keep their existing repair path. - No schema migration, credential change, budget increase or warm-session policy change is included. Normal pause, ownership, budget, approval and dispatch gates still apply. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tool use and code execution. The session does not expose an exact backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; full local run limitation recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-17 22:57:10 -05:00
DottaandPaperclip	e6dcc6b095	Connect personal Muse through secure native execution and Vault OAuth (#16448) ... Add company-scoped Muse native control, OAuth, wake receiver and recovery. Recheck current authority after waits and before effects or receipt replay. Preserve task-read scope, assignment expiry, independent holds, nonblocking reassignment and truthful cleanup provenance. Retain source trust on progress comments and verify current Muse V8 authority through real signed-JWT HTTP requests. Preserve the upstream trimmed ordinary authentication lookup. Refs #16448. Preserve captured canonical queue recovery and concurrent environment preparation while retaining the scoped Muse V8 exclusion. Normal migration 0354 and existing Muse SQL remain unchanged. Fresh current-head Greptile review is 5/5 with the project-write finding addressed. Required CI and independent technical review gate the merge; maintainer override is reserved for formal approval rules. Recheck current Muse project-write authority after all receipt/task/run/admission waits, before creation or replay. Real PostgreSQL regressions reproduce four late effects/replays on the old route; the repaired three-file treatment passes 354 cases with no skips, and the scoped server typecheck passes. Ordinary/Dot behavior is preserved. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-17 22:44:36 -05:00
DottaandPaperclip	2cc7992369	Share the external Runner lifecycle and add strict Muse contracts (#16444) ... Add the external Runner foundation with distinct Dot/Grok V6, canonical home-mode V7 and Muse V8 contracts. Keep generated schemas and profiles synchronized. Bound default question-answer command IDs in a disjoint namespace while preserving existing IDs and durable replay. Preserve canonical migration history and add normally generated, replay-safe Muse schema milestones with literal preview-history fixtures. Refs #16444. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-16 13:46:50 -05:00
DottaandPaperclip	adb69e8f09	feat: default new Codex agents to Paperclip Runner on Linux and macOS (#15683) ... ## Thinking Path > - Paperclip is the open source control plane for agent companies. > - People choose a harness when creating an agent. > - The server should resolve that harness to a qualified runner. > - Codex is the first harness in the staged default rollout. > - Saved agents must retain their execution choices. > - New Codex agents therefore use Paperclip Runner on qualified macOS/Linux targets, with an explicit Legacy override. ## Linked Issues or Issue Description Refs #16028, #15555, #15673, #15422, #15631, #15926, #15932, #16149. The runtime/install prerequisite and release repair are merged. This slice owns Codex creation, setup, configuration and preservation. Other harness defaults remain unchanged. It adds no database migration; incoming master migrations, required maps and regression coverage are retained. ## What Changed - Resolve `runner: auto | paperclip | legacy` at the common creation boundary across onboarding, API/CLI, hires, built-ins, plugins and imports. Freeze pending hires and export resolved execution choices. - Preserve existing runners on upgrades, unrelated edits and import updates, including compatible accounts, workspace settings, Codex effort and intentional clears. - Use the underlying harness picker and branding; put the runner override in Advanced. Use the shared dropdown and update production-component stories and creation contracts. - Support qualified Apple Silicon, Intel Mac, Linux x64 and Linux ARM64. Linux requires GNU libc 2.39+. Disabled, external, unsupported and unqualified combinations retain their applicable Legacy execution. - Test the selected runtime/account/model/environment. Ordinary Connect probes eligible immutable managed targets and refreshes scoped discovery. Creation/discovery do not provision. Auth, dependency, incompatible-setting and cleanup failures remain actionable without changing runners. - Keep setup probes read-only, preserve local/SSH isolation and resolve packaged assets before staging credentials. Bound cleanup reads by the remaining absolute deadline; physically close every owned ingress socket before endpoint release. Unconfirmed cleanup retains recovery state. - Preserve validated completed-onboarding callbacks through startup recovery. First file-producing work belongs to the child task and uses the existing dependency/block handoff and task-scoped registration. - Reuse existing E2E fixtures, artifact checks, reporting and controls. Keep Docker/Cargo/native compilation remote. Paperclip tarballs contain no Codex payloads; official optional npm platform dependencies remain. ## Verification Current tested head: `1c03a8815672d82c264a9b08d4ea1984e8fae107`. **Merge-ready: required CI, fresh review, package qualification, Mac core acceptance and Test task/follow-up acceptance pass.** The existing attachment and native-continuity oracles independently verify both Test downloads, run attribution and unchanged native/provider/workspace identities. The maintainer authorized a code-owner-review exception for this PR only. The first Test setup failure and normal Board completion-review friction are retained below. Actual post-merge packages and private Cloud distribution remain separate qualification gates. - [Current CI](https://github.com/paperclipai/paperclip/actions/runs/38564241625): all 48 ordinary jobs and required verify/E2E pass. Attempt 1 retained six simultaneous host-shutdown cancellations. One failed-only retry passes those six lanes and two aggregates, preserving 40 successes. Actual integration checkout `53104c1` combines this head with master `c80`; it is not relabeled as the branch tree. Fresh exact-head Greptile 5/5, 30 resolved threads, no actionable findings. - [Exact-source package qualification](https://github.com/paperclipai/paperclip/actions/runs/38565287239): nine applicable jobs pass, nine conditional jobs skip. Four remote native producers; Linux x64 clean install; Intel Mac, Apple Silicon and Linux ARM64 native startup/physical cleanup controls. All 18 tarballs/four daemon source/hash/manifest bindings and 343 migrations through 0344 verify. No Codex payload is packed. These hosted tarballs are not a published npm canary. - Normal Apple Silicon installation and credential-free native startup/cleanup pass using the packaged daemon, with no source build or provider turn. Fresh installed DB initialization passes. Upgrade 651a→1c03 preserves all three complete execution configs, accounts, ten recorded run profiles, paused budgets, active incidents, cost summaries and two historical unpriced events. Historical recovery state is retained. - Cleanup regressions: four delayed-read baseline failures become 15 passing deadline/authority controls. No-acknowledgement and prior-error socket ownership regressions become 12 passing controls. Direct runtime/server types, maps: 35 areas / 176 IDs, diff checks and independent reviews pass. Permission/auth/reconnect policy and the 10-second setup budget remain intact. - Existing source coverage includes creation/selection/imports, pending approvals, recovery, account/lifecycle handling, mounted forms, dropdowns and E2E support. Full typechecks/tests/build/token and applicable Storybook checks pass in the recorded current CI; no duplicate suite or weakened grader. - Literal 9546 live Mac proof: ordinary Auto Codex onboarding, useful document, real completed-child callback/reply, seven successful native runs, manual follow-up and independently downloaded1,101-byte child file. Existing Legacy title edit/reload preserves execution; two real Legacy runs produce independently checked15/34-byte files. These remain historical source records. - Current literal 1c03 Mac core acceptance passes independent checks: ordinary Auto native Codex onboarding without runner override; only timeout 180 seconds/concurrency 1 saved; six successful native runs and six serialized turns (maximum overlap 1); both same-task continuity checks and five mechanical callback checks; actual rendered parent reply and exact child downloads. The useful handoff advances from revision 1 to 2. Both files are 48 bytes, SHA-256 `8b742bd4e8123a7c078f0a8d9f3449fbfa8a320d7d90be641a2818db3c53d92c` and `f42db9774fa06e804ab7d596fc8b922a2b638921c418b87d0111151d290a6afe`; exactly one byte changes and old history remains. The agent is paused with zero active runs/errors. Both artifact work products have `isPrimary=false`, and descriptive titles fail a stricter title-helper assumption; these retained results are scoped editorial follow-up, not new merge gates. Six source-bound screenshots are uploaded and verified in the private primary QA report. - [Literal 9546 managed proof](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-38485488621-1/index.html): original 19 assertions and completed judge pass, four native turns, child-bound240-byte file and session/workspace continuity. Checkout reports a verified generated target-lock mutation; full final porcelain was not retained. Signed execution-image identity remains separate from Core source. - Actual 651a Test deployment succeeds on one owned/pinned target. One ordinary Auto API-key Connect then fails native cleanup before hire. Controller turn/drain/suspend completion and service-recorded environment cleanup are observed; they do not certify physical deletion or native authority retirement. Failure/recovery/unknown holds are retained. No Legacy switch or unchanged paid retry. The two source repairs above require actual treatment. - [Current immutable Core preview](https://github.com/paperclipai/paperclip/actions/runs/38570365725) and matching DB/shared artifacts pass. One owned, pinned Test target serves exact `1c03`; identical migration history correctly skips the migrator. Ordinary Auto Codex Connect succeeds and hires native; saved native selection and QA bounds survive reload. The initial child preparation attempt fails before provider work, with actual stdout identifying `renewal_deadline`; its trigger remains unknown and the failure is retained. Normal Board **Try again** and **Resume** recover the child to Done, producing a browser-verified 44-byte file, SHA-256 `0a8458b1efcc2d7a0d815689458c4caab039a46c7c7a3e5805de6bc8b1a96ecc`. The actual parent callback reports the completed child document/file. The existing validator requires a parent-owned attachment, so recovery enters normal Review; after byte checking, normal **Approve completion** reaches parent Done. No forced status or duplicate parent file is used. The validator is unchanged from integrated and current master. Autonomous initial callback-to-Done is not proven. The same-child follow-up completes document revision 2 and publishes a second independently downloaded 44-byte file, SHA-256 `22022257300fae762784632d08d3bbe0bab61e2ce81f91864a14dcd7aa464591`; only the revision digit changes. The unchanged existing native-continuity and attachment oracles pass. All ten recorded runs are terminal: nine native successes and the retained pre-provider setup failure. The QA agent is paused with zero active runs. The pre-existing parent evidence scope mismatch is a separate follow-up, with ordinary completion review exercised rather than a forced status or relaxed validation. This unsigned Core preview remains distinct from post-merge private Cloud composition. ## Risks - Saved agents are not migrated. Explicit runner changes use normal configuration/session rules; historical runs retain their recorded execution. - Native readiness depends on verified assets and server-owned target qualification. A Legacy probe, HTTP200 or successful account save cannot prove native readiness. - Recovery exceptions remain restricted to validated completed-onboarding reports; process-loss, ownership and terminal-state safeguards remain active. Unexecuted live journeys stay open. - Linux ARM paid tasks, Windows, Alpine and 32-bit ARM remain deferred. Cold Cargo cache misses are recorded; no warm-cache speedup is claimed. - Historical invoices/physical cleanup remain unconfirmed and held. After merge, exact published npm canary, signed standard image/migrator, private composition and serving/live acceptance still require qualification. Production deployment/default promotion is not authorized. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, tools, code execution and parallel agents. Exact serving snapshot and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have linked public issues or PRs above - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run the stated focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green on the current candidate - [x] Greptile is 5/5 with no open findings on the current candidate - [x] I will address all reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 11:33:02 -05:00
DottaandPaperclip	ebeac833a6	perf(runs): bound history and reduce repeated context reads (#16286) ... ## Thinking Path > - Paperclip helps operators manage AI agents and inspect their work. > - Run history supplies recent execution records to the board and API clients. > - The list repeats eight reads of a large stored JSON context. > - A caller can also omit the limit and read the complete history. > - This pull request expands context fields once and bounds every list read. > - The benefit is lower database cost and bounded redaction work. ## Linked Issues or Issue Description Refs #958. Related community reports: #13962 and #10631. This is an independent implementation. It does not use their commits or code. It covers the history bounds reported in both. Process memory monitoring from #10631 is outside this change. **What happened?** The run-history query repeatedly extracts fields from large context JSON. Requests without a limit also load every historical run before authorization and redaction. Accumulated staging query statistics showed 271 ms for summary history and another 58 ms for its redaction lookup. **What did you expect to happen?** Normal reads must use a bounded recent history. Context projection must preserve the existing values and privacy behavior with less repeated database work. **Steps to reproduce** Run the disposable benchmark below against the baseline revision. It creates 1,400 synthetic runs with 192 KiB contexts. It compares the exact service SQL and every returned column for both summary and full history. ## What Changed - Expand context fields with one lateral record projection. Preserve non-object legacy contexts and the first-class issue binding. - Enforce a 200-row default and a 1,000-row maximum in the route and service. - Apply the default in the shared UI client, including both agent detail implementations. - Preserve response arrays, authorization, result projection and secret redaction. - Fetch older linked runs by ID in both agent pages. Check company and agent scope, navigation races and visible retry errors. - Update OpenAPI, data-path documentation and capability maps. - Add real PostgreSQL and HTTP regressions plus a reproducible benchmark. ## Verification - Local benchmark: summary history 37.72 ms to 19.34 ms; full history 37.84 ms to 19.01 ms. All 200 complete positional rows match in each comparison. These are synthetic local SQL timings, excluding HTTP and redaction. - History, route, privacy and redaction suites: 160 server tests passed across six files. Twenty-two UI API, selection, run-panel and log-visibility tests passed. Both agent-page variants render an older linked run with an empty history page on desktop and mobile. - `pnpm -r typecheck` and `pnpm build` passed. - Map validation and `git diff --check` passed. - All current-head CI and security checks passed, including the full general and serialized suites, browser shards, Runner verification and build. Greptile reviewed the exact commit at 5/5 with no remaining findings. - The full local sweep began before the final compatibility fixes. It was stopped after the final commits passed complete CI. The focused local suites listed above passed on the final code. Before interruption, the broader local sweep passed 31,743 tests across the general server, UI and several package groups, with no reported failures. The full local command is not claimed as a completed pass. ```sh pnpm --filter @paperclipai/server exec vitest run src/services/heartbeat/run-state.test.ts src/__tests__/heartbeat-list.test.ts src/__tests__/agent-live-run-routes.test.ts src/__tests__/run-secret-redaction.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts pnpm --filter @paperclipai/ui exec vitest run src/api/heartbeats.test.ts pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-run-history.ts --baseline-ref=1e731520bb ``` ## Risks - Clients that relied on an unbounded array now receive 200 rows by default. Explicit limits allow up to 1,000. This change does not add pagination. - Legacy context conversion must stay identical. Real database regressions cover null, array, scalar, nested values and native issue precedence. - No schema migration or persistent cache is introduced. Redaction and access checks remain active. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository inspection, shell tools and local code execution. The runtime does not expose a more specific backend model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-14 10:14:41 -05:00
DottaandPaperclip	b8ffeb973a	feat(runner): add experimental Grok Bot agent participation (#16219) ... ## Thinking Path > - Paperclip manages AI agents that perform work for a company. > - The shared Runner owns task admission, authority, receipts and finalization. > - An existing Grok Bot needs to receive assignments and return results through that Runner. > - The Bot also needs governed access when it starts work from its own conversation. > - This PR adds agent-purpose MCP access and routine webhook dispatch. > - It reuses the external-agent Rust Runner and keeps Grok capabilities explicit. > - The provider stays experimental until the real-client managed journeys pass. ## Linked Issues or Issue Description **Agent or provider** Grok Bot, distinct from the Grok Build coding runtime in #13882. **Why this adapter is useful** Users can assign work to an existing Bot, receive progress, questions, documents and files in Paperclip, and let the Bot initiate work under its agent permissions. **How the agent is invoked** A webhook-only routine delivers bounded references. Dedicated named-agent OAuth and MCP provide inbound calls. Setup follows [Grok routines](https://cursor.com/help/grok-bot/routines) and [custom plugins](https://cursor.com/help/grok-bot/connect-plugins). The current desktop app needs its Bot-generated Webhook URL navigation link when routine-row clicks fail. The GitHub search found related Grok Build runtime work, but no existing Grok Bot agent provider. This extends the roadmap’s existing agent, Runner, governance and artifact foundations. ## What Changed - Add default-off Grok Bot participation through the shared real Rust external backend, preserving native provider identity guarantees and distinguishing session closure from remote termination. - Bind agent OAuth to company, agent, client, gateway and authorizer; recheck permissions and task-bound execution authority without manufacturing a native identity. - Persist routine configuration, dispatch intents, authenticated acknowledgement and recovery holds. Store URL/key together in encrypted secret storage, use guarded public egress, and never automatically replay ambiguous HTTP delivery. - Reuse native tools, durable waiting, completion assessment and review. Add task-bound file grants through existing artifact storage; suppress proactive echo dispatch and synchronous result wake loops. - Enforce the current native deliverable work-mode policy at file-ticket issuance and use. Add operator retries for confirmed rejected wakes, with retained receipts, stable retry intent and current-authority checks. Recovery locks bindings before routine rows. Thread Stop resolves the selected run and its agent. - Cancel queued and preparing Stop targets before assignment creation, restore saved setup after a failed request and successful retry, and publish the Grok provider descriptor schema. Advance the versioned Copilot source declaration for the shared schema change; its qualification remains pending. - Filter task summaries with the shared privacy policy, preserve feedback admission when wake delivery is blocked, reconcile settled delivery receipts without restoring work authority, and bind every Stop control to its displayed run. Recovery records distinguish dispatch uncertainty from a revoked session. - Add resumable guided setup, credential replacement, consent, management, observed work status and stop/recovery controls, plus an interactive Storybook journey using the production task thread, question form and report card. Grok’s own apps remain available; Paperclip workspace and third-party tools are deferred. ## Verification - Five focused protocol, authorization, person MCP and Runner files: 192 tests pass. The Grok assigned-work case executes the actual Rust Runner, saves a document, uploads/deduplicates a file and reaches ordinary finalization. - Proactive intake/deduplication and renewed-session recovery pass. These use scripted clients and synthetic storage; they are not real-provider acceptance. - Repository typecheck, production build, token gates, map checks and Storybook build/catalog checks pass. The local full-suite campaign was stopped after macOS shared-memory exhaustion repeatedly prevented embedded PostgreSQL startup. No local aggregate pass is claimed. The first clean-worker CI run identified avatar fixture/configuration, flag defaults, feature-card ordering and OpenAPI route-coverage gaps; all four focused regressions pass after correction. Those regressions pass. The latest clean-worker CI run also found a Copilot source-hash declaration mismatch after the shared descriptor update; the corrected declaration passes all 28 integrity tests. The next review corrections pass 144 rendered UI tests, 21 OpenAPI tests and 82 Runner contract/integrity/history tests. The latest database fixtures were blocked at startup by host shared-memory exhaustion; clean-worker CI must execute their new regressions. Full current-head CI now passes: 54 successful checks and four intentional skips. The database regression uses the PostgreSQL driver result shape; file-policy, rejected-feedback and lock-order cases pass on clean workers. Current-head Greptile is 5/5 with no unresolved findings. - Rendered setup steps and outbound-only verification were walked through in Storybook. Simulation cannot enable Done without an authenticated-reply state. - Actual client OAuth, continued access, reconnect and one routine wake with an exact chat acknowledgement were observed separately. This candidate’s live authenticated challenge, assigned/proactive workflows, Linux managed artifact, restart and cleanup are not yet qualified. The provider stays disabled by default. - The separate Cloud ingress change forwards only exact task-bound upload/download paths and preserves host identity and header stripping. - Current Cloud policy reserves Test for pre-merge app/harness experiments; Staging accepts only master-derived revisions. Real-client qualification needs a supported, authorized deployment target and remains outstanding. ## Risks One append-only migration adds routine/dispatch state and extends shared bindings and file tickets. The unshipped migrations were regenerated after upstream consumed their original numbers. Rolling back code retains the durable rows; revoke grants and inspect unknown remote activity before resuming dispatch. Grok cannot confirm remote termination, stream its complete transcript or report spending. Plugins are account-wide and do not isolate credentials between Bots. Real-client managed acceptance remains required before enabling the experimental provider. ## Model Used OpenAI Codex based on GPT-6, with reasoning, code execution and browser/native UI tools. The exact deployment model ID and context-window size are not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-13 20:57:42 -05:00
4 people	0b0e53029f	fix: recover saved workspace results automatically (#15824) ... ## Thinking Path - Paperclip preserves accepted agent results before workspace copyback and result commitment. - A replaced controller can leave a physical copyback owner receipt behind. A lost database lock or expired lease cannot prove that its tar/Git children stopped. - Infrastructure recovery should belong to the runtime and deployment harness; task users should not investigate processes or attest shutdown. - New receipts record the exact deployment. The harness independently verifies that deployment fully stopped and signs a short-lived proof bound to the stack, run, owner and deployment. - The existing guarded reconciler then finishes the same accepted result without a new agent turn. Legacy receipts retain the documented administrator API after independent platform verification. ## Linked Issues or Issue Description **What happened?** An accepted native result can remain at `workspace_finalizing` with an abandoned physical copyback owner. Repeated recovery sweeps could replace specific instructions with advice to start new agent work. The first version of this PR exposed a shutdown-evidence form in the task; that asks end users to resolve infrastructure details. **Expected behavior** Automatically verify and recover the exact stopped deployment, preserve the original accepted result and existing approval/monitor waits, and show only a quiet task status. Unknown or live workers must remain fenced. Older receipts without platform identity need independently verified operator assistance through the administrator API. **Steps to reproduce** 1. Persist an accepted native result and a copyback receipt from a prior controller. 2. Replace that deployment before workspace finalization commits. 3. Let the normal finalization reconciler revisit the saved result. 4. With a platform-bound receipt and a fully stopped deployment, recovery should finish automatically. With missing, live, unverifiable or changed ownership, it must not write files or create another provider turn. Related public work: #14402 introduced copyback ownership; #9687 handles dead-run finalize barriers; #15795 handles terminal stale-run cleanup. This change addresses an accepted result whose physical copyback owner prevents commitment. ## What Changed - Record Railway project, environment, service and deployment identity on new physical copyback owners. - Request read-only platform verification from the managed cloud harness. Require an Ed25519 stop receipt with a disjoint purpose, configured signing key, exact stack/run/owner/deployment binding and valid expiry. - Keep live, unknown, malformed, unavailable and legacy owners fenced. Compare the whole prior receipt during ownership takeover, under the existing dedicated advisory lock. - Persist platform proof and audit automatic takeover. Resume normal finalization of the same saved result, preserving task ownership, approved boundaries, monitors and pending confirmations. - Remove process metadata, shutdown evidence, confirmation checkboxes and infrastructure-repair forms from task views. Suppress the duplicate technical owner-recovery card. Show “Restoring saved work…” only from server-confirmed recovery or queued operator recovery; platform identity alone shows the operator-assistance fallback. Remove the status when recovery resolves. - Share current-task, current-running-execution, accepted-result, authoritative-reviewer and newer-execution checks between automatic and operator recovery. Validate after the provider lookup, atomically with the owner claim, and again before file publication; closed or superseded tasks cannot start copyback. - Keep the separately authorized exact-owner API for operator recovery of legacy receipts, with active-lock, live-controller, stale-result and newer-execution rejection. Generic dismissal cannot bypass the copyback fence. - Replace obsolete form stories with production-backed automatic, queued, legacy, completed and mobile examples; retain stable suite IDs. Update the operator runbook and scoped Storybook catalog. ## Verification - Red/green: new platform-proof tests failed before implementation; the database takeover regression failed against the original ownership guard; reduced-UI assertions failed against the original form. All pass after the changes. - Passed: 59 physical-owner/finalization database tests using isolated PostgreSQL, including real reconciler commitment from all three saved-result phases through both automatic and operator recovery. They preserve result identity, exact saved file bytes, pending approval and overdue monitor waits, and create no new run or wake request. Unknown evidence and changed ownership remain fenced. - Passed: 162 focused proof, cloud identity, recovery route, recovery-card and reduced-UI tests. - Passed: full workspace typecheck and build, final server/UI typechecks, token gates, and Storybook build. - Browser inspection: desktop task composition shows only the quiet status; completed example removes it and retains the approval-wait message. Mobile shows the existing bottom navigation with no horizontal overflow. These are fixture UI observations, not live provider acceptance. - A companion harness change implements the actual provider check, signed receipts, server-token authentication and bounded/coalesced provider reads. Its full typecheck and test suite passed: 3,240 passed, 92 skipped, zero failures. This application alone cannot enable platform verification. - The updated full application `pnpm test:run` was stopped after repeated local embedded-Postgres startup failures; it is not claimed green. The same affected recovery suites pass against isolated PostgreSQL. CI passed on final head `9ad5c3529e`: 54 successful checks, two intentional skips, and successful legacy Snyk status. Greptile is 5/5 with no unresolved findings. - Global Storybook catalog validation is green: 225 files, 2,224 stories. Its 17 catalog/index tests pass. Master was merged and the inherited index metadata was reconciled after source review. - Review correction red/green: two UI and two database assertions reproduced misleading progress / missing server recovery state; all pass after server-confirmed state was added. All 153 focused server tests and 68 UI tests pass on the final recovery code, including repeated-sweep idempotence. Full workspace typecheck/build, token gates and Storybook build also pass. - Safety regression red/green: stale-task copyback and publication tests failed against the previous guard. The final suite covers changes during proof lookup, closed/cancelled tasks, replaced execution/checkout, newer and queued legacy runs, reassignment and rejected/unbound results, plus the real reconciler selecting closed or superseded tasks. Files remain untouched and no workspace operation starts. - No staging mutations, deployment, live provider probes or live production recovery were performed. The three historical receipts lack deployment identity and still require independent operator proof. ## Risks Automatic takeover requires the companion harness release and configured runtime-identity keys. Deploying only this application leaves unavailable evidence fenced. Legacy receipts are not retrospectively assigned a deployment identity. An authorized operator can still make an incorrect attestation through the fallback API; its administrator gate and exact-owner safeguards remain. Provider history that is missing or deeper than the bounded lookup defers automatic recovery. Copyback can still fail under its existing retained-workspace policy. No migration, telemetry change or cloud provider credential delivery is required. The companion harness PR also has passing CI and a 5/5 review; neither branch is deployed. ## Model Used OpenAI Codex, GPT-6, with repository inspection, browser verification, tool use and code execution. The exact serving model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the relevant tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing>	2026-10-13 10:14:45 -05:00
DottaandPaperclip	be380ba155	fix(ui): hide agent-created tasks by default (#16157) ... Hide agent-created, system-created, and chat-channel tasks from independent collections by default. Preserve task hierarchy context and existing inbox participation. Add a saved opt-in checkbox without helper text. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-13 09:26:37 -05:00
DottaandPaperclip	f329b24ee1	feat: give tasks durable project-optional workspaces (#15749) ... Give tasks durable company-scoped workspace bindings independent of project assignment, channel defaults and authorized repository preparation. Reuse verified input seeds and sparse output recovery while preserving trust, privacy and controller provisioning boundaries. Full CI and E2E, Superagent, legacy checks and Greptile Apex 5/5 passed on reviewed head 753680b61c. All review findings are resolved; the PR records focused verification and historical acceptance limits. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-12 20:45:18 -05:00
3 people	1b4c803816	fix(costs): preserve usage attribution and explain unavailable run costs (#15948) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The cost ledger records usage and controls agent budgets. > - Some runs lose the selected model or API billing category before their usage reaches the ledger. > - The dashboard reports missing costs but does not identify the affected runs or the available actions. > - This pull request preserves usage attribution and adds a compact view of cost gaps grouped by agent. > - Operators can inspect runs, open agent settings, and record saved costs without starting new agent work. ## Linked Issues or Issue Description Refs #15634, #11938, #15879. This follows the merged subscription reporting in #15634. The older proposal in #11938 addresses part of ACP model attribution. This change also rejects conflicting models across a turn, fixes credential-backed billing capture, and explains cost gaps in the existing dashboard. The Pi receipt recovery work in #15879 covers a separate adapter recovery path. This PR preserves unresolved receipts and does not recover provider usage or reprice past entries. **What happened?** ACP results use the requested model even when the runtime selects a model. Native and OpenCode API runs can report unknown billing despite a matching API credential. The dashboard shows notices without links to the affected runs. These gaps make API and subscription totals harder to interpret. **Expected behavior** Record the model selected by the runtime. Identify API billing only from a matching credential. Keep costs unresolved when reliable usage or pricing is absent. Show the affected agents and runs. Offer an accounting retry only when saved evidence is ready. **Steps to reproduce** 1. Run an ACP agent with its model left unset, or use a matching API credential with a native or OpenCode agent. 2. Inspect the recorded model and billing category for the run. 3. Open Costs Overview for a period with unpriced usage or terminal runs awaiting receipts. 4. Inspect the notice and try to find the affected run or the relevant agent settings. **Paperclip version or commit** Current master at `657e71584a0`. The patch is rebased on that commit. **Deployment mode** Self-hosted server and local development. The cost detail tests use a disposable PostgreSQL database. ## What Changed - Preserve the runtime-selected ACP model in checkpoints and final results. Conflicting models across one turn remain unpriced. - Share credential-backed API billing checks across native and OpenCode results. Keep unknown billing when the provider and credential do not match. - Add company-scoped cost detail and accounting retry endpoints. Board viewers can read details but cannot retry. Agents and foreign companies are denied. - Limit each detail category to 25 entries. Use one consistent database snapshot and sparse indexes matching the filters and sort order. Add the unknown-billing count to the existing summary query. - Replace the notice with an expandable count line. Group entries alphabetically by agent. Show the saved blob icon, link each name to configuration, and put run and retry links beside the date. - Preserve estimate and unavailable-price labels on Providers and Billers; keep retry confirmation after the resolved notice disappears. - Fetch details only while expanded. Retain the last loaded details after a refresh error. Retry saved accounting only; do not rerun the agent. - Add capture, database, authorization, UI, and Storybook coverage. Update the implementation contract and cost reporting guide. ## Verification - `pnpm -r typecheck` passed after rebase. - `pnpm build` and `pnpm build-storybook` passed after rebase. - `pnpm check:token-gates` and `git diff --check` passed. - 1,086 focused tests passed across 17 files. These cover ACP, native and OpenCode capture, PostgreSQL reports, route authorization, OpenAPI, Costs UI, and existing accounting regressions. - Desktop and mobile browser checks passed during UI review. They cover lazy expansion, alphabetical groups, loaded blob icons, configuration and run links, keyboard retry, and layout overflow. - `pnpm test:run` was attempted with command-scoped `init.defaultBranch=master` for Git fixtures. It was stopped after a native session recovery failure reproduced on unchanged base `657e71584a0` (`archives a damaged prior epoch before completing guarded same-task replacement with real runnerd`). An MCP upload case failed in the broad run but passed in isolation on both the base and PR branch. These local limitations do not affect the 1,086 passing focused tests. - Current-head CI is green, including all unit test lanes, serialized server suites, runner checks, typecheck, build, policy, and eight browser shards. Apex re-review is 5/5 on `56de60effdb`; all four original findings are addressed and every review thread is resolved. - PostgreSQL applies migration `0333` in the integration fixtures. EXPLAIN ANALYZE on the production detail statements uses the new sparse indexes behind 5,000 healthy records in each table. - Ten provider/biller card cases cover missing and estimated prices; the retry test covers row removal and notice unmount. A live Storybook browser check verifies the confirmation and updated count. - Run `pnpm storybook`. Open **Costs / Subscriptions / Cost Data Quality Details** to inspect the production page with representative fixtures. ## Risks - Model and billing metadata change for future runs. Estimates still require complete usage and a supported price. No past cost amount is rewritten. - A missing final receipt remains unresolved for budget checks. The display changes after fifteen minutes; that time limit never authorizes settlement. - Accounting retry is a write action. It retains the existing company checks, receipt fences, idempotency, and audit logging. - Detail reads are bounded and run only after expansion. They poll every 30 seconds while open. No provider probe or new summary query is added. - Migration `0333` adds two partial indexes; it changes no stored cost amounts or table columns. Index creation scans the existing tables and may briefly block writes during deployment. No lockfile or workflow change is included. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code editing, browser checks, and test execution. The exact serving model ID, reasoning setting, and context window were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-11 22:48:29 -07:00
DottaandPaperclip	a7dd5284b5	feat: add company fast-response acknowledgements (#15748) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Human messages start agent work through task, chat, and external-channel intake. > - Agent startup can delay the first visible response. > - A small direct model call can acknowledge the request while normal work starts. > - This response must remain separate from evidence that the agent answered or completed work. > - This pull request adds optional company-wide fast responses with durable receipts, access checks, and cost accounting. > - Users receive prompt acknowledgement without changing the agent execution path. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: database, shared contracts, server orchestration, connections, and UI. **Problem or motivation** Users can wait for an agent to start before they receive a contextual response. Generic progress messages do not confirm what the user asked. An acknowledgement must not falsely complete work or satisfy an unanswered turn. **Proposed solution** Add a disabled-by-default experimental fast-response model using an existing shared API connection. Generate one short acknowledgement for each accepted human turn. Use the existing agent identity and external publication outboxes. Retain platform provenance and account for the inference independently. **Alternatives considered** Running the acknowledgement through the agent runtime adds startup latency. Restricting implementation to OpenRouter leaves other existing API connections unsupported. This change supports all existing API connections and uses OpenRouter for live verification. **Roadmap alignment** This extends existing agent communication and connection settings. ROADMAP.md and open pull requests were checked. No duplicate fast-response implementation was found. The maintainer supplied the feature plan directly. ## What Changed - Add Experimental fast responses to instance settings. Keep the feature off by default. Enforce the flag at intake, dispatch, and publication. - Move Decision model and Fast response setup to company Settings → Connections when the experiment is enabled. Place Connections below Environments. Separate the model sections with clear headings and a divider. Keep Decision model in General when the experiment is off. Route Costs setup links to the available settings page. - Make the acknowledgement prompt a multiline template string. Ask for a brief, positive response in the user’s language. - Add settings, connection/model selection, sponsorship, fixed-sample testing, and Costs history. - Add a Vercel AI SDK text service for existing native, routed, Bedrock, and local API connections. - Add durable per-turn requests, a bounded worker, a three-second admission deadline, and one inference attempt. - Integrate task creation, task and agent chat, steering, queued follow-ups, experimental board chat, shared chat intake, and email. - Preserve receipt provenance and exclude receipts from completion, recovery, interaction resolution, and agent wakeups. - Recheck authorization, source validity, real replies, budgets, and destination binding before publication. Preserve accounting for suppressed or uncertain outcomes. - Add provider contracts, database race tests, channel outbox tests, UI tests, Stories, and documentation. Keep decision-model execution unchanged. ## Verification - Local `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after rebase. The UI typecheck and build also pass after the final navigation fixes. All 37 focused navigation tests pass. The first full local `pnpm test:run` invocation reported seven chat failures. All seven passed in isolated reruns after fixture updates and host cleanup. This invocation was stopped after the full final-head CI suite passed. - Initial local database tests hit macOS shared-memory exhaustion. A diagnostic confirmed `shmget: No space left on device` with the system limit of 32 IDs. Three abandoned, unattached PostgreSQL IPC headers were reclaimed. The isolated rerun passed all 31 fast-response and channel tests. The two additional Slack regression cases also passed in a separate rerun. No active database was stopped. - UI token gates pass. Provider contract tests cover all supported API connection types. Fast-response tests cover uniqueness, authorization, budgets, expiry, cancellation, restart, real-reply races, and the experimental flag. - The test drive verified that disabling the experiment hides Connections and redirects its direct URL. Enabling it restores the saved OpenRouter setup. After restart on the final commit, the OpenRouter settings sample returned “Sure, I’ll take care of fixing the border around the settings panel.” in 428 ms. It used 260 input and 31 output tokens and recorded $0.0000576 in reported cost. - Earlier live OpenRouter verification produced a receipt in 1.013 seconds for a new task. The real agent then supplied its substantive answer. A separate settings test took 1.34 seconds, used 246 input and 23 output tokens, and recorded $0.0000361. - Live Slack verification on final commit `24016fa525961ebc7a4acffdc5019ce123970549` passed for a fresh initial task and a follow-up in a disposable bot DM. The initial acknowledgement appeared 2.405 seconds after the Slack message and 1.413 seconds after durable acceptance; the follow-up appeared in 2.322 seconds and 1.160 seconds respectively. The recorded generation/settlement durations were 898 ms and 697 ms. The initial call used 291 input / 39 output tokens and cost $0.00006705; the follow-up used 281 / 47 and cost $0.00007035. - The Slack acknowledgements retained the assigned agent identity and `origin: fast_response`, source-turn references, and no fabricated run ID. A status snapshot after the initial receipt showed the task still `in_progress` with its real execution run. The agent then read the CSV fixture and correctly reported two ready items, two blockers, and their owners. Both replies stayed in the originating DM, and no duplicate generic working message followed the delivered contextual acknowledgements. The `new` command started a fresh conversation session without generating a fast response. - An earlier eligible Slack turn generated a response but suppressed publication. Its reported cost was retained, generic progress remained, and the real agent completed the requested result. This is recorded separately from the two delivered acknowledgements; the history did not expose a precise suppression reason. - Slack verification used real signed provider events, the existing intake/publication paths, and a user-approved temporary webhook-only HTTPS tunnel. Subscriptions were limited to direct-message events. The test bot was paused and the tunnel stopped afterward. Live channel/group-thread and GitHub journeys remain unverified; their mocked tests cover intake, destination binding, publication, and continued underlying work. - Greptile reviewed final commit `24016fa525961ebc7a4acffdc5019ce123970549` at 5/5. Both new review findings are fixed and all threads are resolved. [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/38060015730) passed. This includes grouped server/chat/workspace tests, serialized suites, runner checks, browser journeys, typecheck, build, release registry, policy, and the canary dry run. - Manual check: enable Experimental fast responses. Open company Settings → Connections. Select a shared API connection and model. Save and use Test response. Submit a task message and confirm that its receipt does not complete the task. ## Risks - The migration adds receipt and configuration tables plus accounting references. It uses idempotent statements and follows the current migration sequence. - A slow provider, authorization change, or real-agent reply can suppress an acknowledgement. Existing progress feedback remains available. - Unknown provider billing retains a reservation until reconciliation. External deliveries with uncertain outcomes must not be blindly repeated. - Live Slack initial-task and follow-up DM journeys pass. Live Slack channel/group-thread and GitHub journeys remain unverified. The PR remains a draft and records these gaps. Final-head repository CI and automated review pass. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser interaction. The runtime does not expose an exact model snapshot ID or context-window size; these are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-10 13:09:17 -05:00
3 people	32745f4e84	feat(issues): keep follow-up parents and show the creation source (#15762) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents delegate work by creating follow-up tasks through the issue API, the CLI, and the chat `create_task` tool > - An agent can omit `parentId` when it creates a follow-up from an ordinary execution task. The API accepts the task as a top-level task. The task hierarchy does not show where the task came from, even though the run provenance is recorded > - The operator must read run history to learn why the task exists. This is slow and it hides delegated work from the parent task > - This pull request keeps the parent link for delegated follow-ups by default, keeps intentional standalone tasks possible, and shows an access-checked "Created from <task> by <agent>" note on the task page from the recorded run provenance > - The benefit is that each agent-created task shows its source without log inspection, while structural parentage, provenance, and blockers stay separate concepts ## Linked Issues or Issue Description Refs #13241 (shows tasks created from a task on the source task page; this PR shows the reverse direction on the created task) Refs #9727 (open proposal for creation provenance tracking) **What existing behavior does this improve?** Issue creation by agent runs, and the task detail page. **Subsystem affected** Server (`POST /api/companies/:companyId/issues`, `GET /api/issues/:id`), shared types, UI task detail page, CLI `issue create`, Paperclip skill docs. **Current behavior** When an agent run bound to an execution task creates a task and omits `parentId`, the task is created with no parent. The task page shows no source. The operator must inspect the run log to find the task that the agent was working on. The source run and the privacy parent are recorded, but the UI does not use them. **Proposed behavior** - When an agent run bound to an ordinary execution task omits `parentId`, the server sets `parentId` to that task. The `issue.created` activity records `parentDefaultedFromRunIssue: true`. - An explicit `parentId: null` keeps the task standalone. Conversation runs (chat handoffs), board or user creates, watchdog follow-ups, a run task the agent may not mutate, a default parent that would cause a delegation cycle, and a default parent the agent may not create a child under (for example a protected assignment policy without a grant) also stay standalone. The server runs the same `tasks:assign` decision with the defaulted parent in scope before it accepts the default. No request that succeeded before now fails. An explicit `parentId` is still checked as before. - `GET /api/issues/:id` returns `createdFrom` with the source task, the run, and the agent. The server resolves it from `origin_run_id`, or from the recorded creation activity for historical rows. It returns `null` when the viewer may not read the source task or the run, when the run is from another company, or when provenance is missing. - The task page shows "Created from <task> by <agent>" in the classic breadcrumb area and in the streamlined Tasks tab. The note is not a blocker and it is shown with or without a parent. - The CLI `issue create` command gets `--standalone`, which sends `parentId: null`. **Reason and benefit** Operators see why a task exists on the task itself. Delegated follow-ups stay inside the parent task tree, so parent completion and child wake-ups keep working. Private and cross-company sources stay hidden. **Breaking changes** None for the API schema. Agents that create tasks from a run and relied on the omitted `parentId` to mean "top-level" must now send `parentId: null` (CLI: `--standalone`). The skill and the spec document this. ## What Changed - `server/src/services/issue-creation-source.ts` (new): resolves a task's creation source from the origin run or the creation activity, scoped to the company, and loads the run-bound task by UUID or identifier. - `server/src/routes/issues.ts`: `POST /companies/:companyId/issues` defaults `parentId` to the run's task when the body omits it (with the exclusions above) and records the default in the activity log. `GET /issues/:id` adds the access-checked `createdFrom` field. - `packages/shared/src/types/issue.ts` and index exports: new `IssueCreationSource` type and the optional `createdFrom` field on `Issue`. - `ui/src/components/task-detail/IssueCreatedFromNote.tsx` (new), `TaskDetailTasksPanel.tsx`, and `IssueDetail.tsx`: show the creation source in both task page layouts and count it in the Tasks tab badge. - `cli/src/commands/client/issue.ts`: `issue create --standalone`. - Docs: `doc/SPEC-implementation.md`, `doc/CLI.md`, `skills/paperclip/SKILL.md`. - Tests: `server/src/__tests__/issue-creation-source-routes.test.ts` (new), `TaskDetailTasksPanel.test.tsx`, `IssueDetail.test.tsx`. ## Verification Run the new server route test (embedded Postgres): ```sh cd server && pnpm exec vitest run src/__tests__/issue-creation-source-routes.test.ts ``` It covers: ordinary delegation gets the parent and `createdFrom`; explicit `parentId: null` stays standalone but still shows the source; conversation handoff and board create stay standalone; a would-be delegation cycle falls back to standalone; a protected source task without a grant falls back to standalone while an explicit parent still returns 403; an explicit cycle still returns 409; historical provenance from the creation activity; missing provenance, a private source for an outsider, and a cross-company run all return `null`. Related existing suites that passed locally (11 files, 313 tests): `issue-created-from-routes`, `issue-create-deduplication-routes`, `issue-agent-mutation-ownership-routes`, `chat-project-tools`, `issue-assignee-invokability-routes`, `issue-watchdogs-routes`, `issue-dependency-wakeups-routes`, `public-mcp`, `native-runtime/task-creation-backlog`, `issue-access-grants-routes`, `issue-list-assignee-filter-routes`. UI: ```sh cd ui && pnpm exec vitest run src/components/task-detail/TaskDetailTasksPanel.test.tsx src/pages/IssueDetail.test.tsx pnpm check:token-gates ``` Manual check: as an agent run bound to a task, `POST /api/companies/:companyId/issues` with only a title. The response has `parentId` set to the run's task. Open the new task; the page shows "Created from <task> by <agent>". The related server suites pass locally (`issue-created-from`, `issue-create-deduplication`, `issue-agent-mutation-ownership`, `chat-project-tools`, `issue-watchdogs`, `public-mcp`, `task-creation-backlog`, `issue-access-grants`, `permissions-upgrade-boundary`). The full `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` were not run locally because the sandbox has a 4 GB memory limit and the TypeScript compiler is killed on the server project; the CI gates on this PR cover them. ## Risks - Behavior shift for agent clients: a run-created task without `parentId` now becomes a child of the run's task. Clients that want a top-level task must send `parentId: null`. The watchdog, conversation, and board paths are excluded, and the default never adds a blocker. - The default parent inherits the parent's execution workspace, which run-created tasks already inherited from the run's task. - `createdFrom` adds one run lookup, one issue lookup, one access decision, and one agent lookup to the task detail request. It is skipped when the task has no origin run and no creation activity with a run. - Privacy: the field returns `null` unless the viewer can read the source task and the run, so a private source is indistinguishable from missing provenance. ## Model Used Claude Fable 5.1 (`claude-fable-5-1`), Anthropic, run through Claude Code in a Paperclip-managed execution workspace with tool use, shell execution, and a local embedded Postgres for tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-10-10 00:40:08 -07:00
DottaandPaperclip	3ec7309d91	feat(voice): add experimental Speko task connection runtime (#15727) ... First PR: runtime. UI: #15728. Qualification: #15729. ## Thinking Path > - Paperclip manages agent work through tasks and governed connections. > - Chat connections already bind external conversations to those tasks. > - Speko handles voice and telephony, while the selected Paperclip agent does the work. > - A call needs durable session authority, ordered follow-ups, and safe result delivery. > - This PR adds an explicit voice transport to the existing connection runtime. > - The UI and qualification suites follow in linked PRs. Speko stays experimental. ## Linked Issues or Issue Description **Problem or motivation** Phone and browser voice conversations cannot use the chat connection task lifecycle. A separate voice runtime is needed to preserve task authority, retries, and reply history. **Proposed solution** Add company-scoped voice sessions, signed callbacks, durable replies, inbound intake, and an agent-facing outbound-call tool. Reuse existing tasks, queues, secrets, and publications. **Roadmap alignment** This extends the experimental chat connector entry points under Agent Chat and Connected Apps. The project owner requested this feature. Refs #11418 for the original provider catalog proposal. This implementation uses the newer chat connection infrastructure rather than an MCP catalog-only integration. Refs #15721 for remote OpenCode startup. ## What Changed - Add additive voice schema, shared contracts, session APIs, and provider callbacks. - Bind browser, inbound, and outbound sessions to an endpoint, task, and selected agent. - Add low-trust guest intake in an explicitly selected sandbox and keep ordinary approval rules. - Submit task work through session-scoped tools and push approved replies through Speko POST /v1/calls/{id}/messages. - Distinguish provider acceptance from confirmed spoken playback. Reconcile reports without downloading recordings. - Add protocol, migration, persistence, provider, and delivery-diagnostic tests. ## Verification - Master 3389bea708 is integrated. Database/shared builds, server typecheck, migration upgrade/replay checks, and 46 communication-guidance/provider/protocol tests passed. - The final persistence suite passed 85 real-database tests. Five focused native continuation/authorization regressions passed. Five callback ownership/rotation tests passed after the last runtime repair. - Review repairs cover reconnect ownership, current caller authority, duplicate delivery, multipart answers, task-page answers, guest question handling, and saved sandbox recovery. - A real OpenCode/OpenRouter DeepSeek task ran in Daytona and saved a correct disk-space answer. This did not establish live-call playback. - Full local typecheck/build passed before the final repairs; focused typechecks passed after them. Latest-head CI runs the repository gates. - The separate integrated browser qualification passed two journeys and failed browser answer retrieval. The feature remains experimental. The report in #15729 records this failure and outstanding real-provider acceptance. - No lockfiles are included. Speko migrations follow master as 0324–0330. ## Risks - Voice sessions add migrations and provider-dependent lifecycle behavior. Migrations preserve existing chat and AI transports. - Callback origins must be reachable HTTPS. Provider keys and signing secrets remain vaulted. - Unverified callers receive task-scoped low-trust authority. Caller ID does not grant private task access. - Outbound telephony incurs provider charges. Provider acceptance is not proof that the caller heard an answer. - Keep the feature behind the existing experimental chat setting until the real browser, inbound, outbound, and delayed-result acceptance matrix passes. ## Model Used OpenAI Codex, GPT-6, with tool use and code execution. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 21:54:11 -05:00
DottaandPaperclip	ed2c22ea8c	feat(connections): simplify dedicated GitHub App setup (#15416) ... ## Thinking Path > - Paperclip manages agent work and the tools each agent can use. > - GitHub bots need their own App identity and restricted repository access. > - Manual App configuration and repeated setup gates make that connection hard to finish. > - GitHub can create an App from a manifest, but people must approve creation and repository access. > - Each Paperclip stack should own configuration, credentials, signature verification, and task admission. > - This change uses Cloud as an encrypted callback and event gateway, then completes setup in the stack. > - GitHub review bots have their own experimental setting. Chat connector opt-in does not control them. ## Linked Issues or Issue Description **What existing behavior does this improve?** Dedicated GitHub App setup, bot settings, and task-scoped GitHub replies. **Current behavior** Users must configure an App, copy credentials, repeat repository selection, and complete manual verification gates. Review settings are hard to scan. Mentions in descriptions can be missed, and automatic completion text can produce an extra GitHub comment. **Proposed behavior** Choose an agent and connect GitHub under a personal account or organization. GitHub asks for creation and installation consent. Paperclip resumes the same draft, stores secrets locally, imports access, verifies delivery, and opens Settings. Each accepted request has one editable App-owned comment for progress and results. Review assessments produce deterministic checks for the reviewed commit. **Breaking changes** GitHub bots require their own default-off experimental opt-in. The setup-method and delegated-agent screens are removed. Per-person automatic-run flags remain readable but are replaced by author filters. Existing connections, direct webhooks, saved restrictions, and manual existing-App recovery remain supported. Companion gateway: https://github.com/paperclipai/paperclip-cloud/pull/688 (merged). The roadmap and related PRs were checked; no duplicate dedicated-App wizard was found. ## What Changed - Add an independent **GitHub review bots** setting across the catalog, routes, identity admission, and task controls. Existing bots continue running when the UI setting is off. - Replace setup with Choose agent and Connect GitHub. Support personal and organization owners, inline low-trust conversion, saved drafts, known organization suggestions, automatic completion, and same-App recovery. - Keep manifests, OAuth PKCE, credential exchange, vault storage, installation verification, and original webhook-signature validation in the stack. Use the Cloud sealed gateway for public callbacks and outbound local delivery. - Add resumable owner metadata in migration 0323. Retry a failed registration renewal through the same durable replacement rather than creating another App route. - Support authorized mentions in issue and PR bodies, comments, inline comments, and review summaries. Preserve attachments, edited-source identity, duplicate protection, and lifecycle invalidation. Filter requests explicitly addressed to another connected bot. - Give agents task-scoped review, comment, and update tools. Update one working comment through progress and final output. Close coalesced requests and recover delayed acknowledgements, including deleted earlier working comments. Preserve automatic replies for legacy connections without saved review configuration. Preserve partial publication evidence and quarantine uncertain automatic fallbacks without blocking Board messages or questions. - Reorganize Settings and Access with rich slash-skill instructions, numeric passing scores, clear automation controls, formal-review tooltips, member and contributor sections, and verified App branding links. Preserve invalid score drafts and automation choices across tabs. - Paginate repositories in 20-row batches with search and a true connection-wide toggle. Batch verification at GitHub's 500-ID token limit. Keep saved repository overrides while hiding their editor. - Give each review a detail URL that remains usable beyond the newest 100 records. Synchronize the manager OpenAPI contracts and add recovery, authorization, UI, and browser regressions. - Reconcile the current master branch and retain its agent avatar, feature catalog, and extracted heartbeat preparation code. Preserve a saved test-drive port and recover from an occupied high port. ## Verification - Focused UI, independent experimental-setting, and CLI tests: 199 passed. - `pnpm -r typecheck`, `pnpm build`, and `pnpm check:token-gates` passed. The broad local Vitest attempt was interrupted after resource timeouts and stale expectations; the affected regressions passed focused reruns. The latest-head full CI matrix passed, including all browser shards, server/chat suites, builds, typechecking, canary packaging, and Docker context integrity. A Docker Hub 429 required a CI retry; it passed without changing the code. - Final review repairs: 127 attachment/event tests and server typecheck passed. All 13 final integration regressions passed under the normal CI timeout, covering legacy reply compatibility, deleted coalesced comments, inline-only review completion, progress edits, and native final recovery. Restart-safe description/review attachments passed the attachment suite. - Final regressions: four reply/lifecycle tests, both native GitHub wait tests, 171 feature-gate/reply-contract tests, the routed review preview, and five scheduler/reply regressions passed. The final fixture repair also passed all six setup-continuation cases and all three complete chat-test shards in CI, covering terminal receipts and retries under the normal CI timeout. One loaded-host receipt run exceeded its local timeout; the exact case passed in an isolated rerun and the complete CI shard. - GitHub integration tests cover personal and organization setup, retry recovery, edited mentions, attachment provenance, coalesced requests, delayed acknowledgements, partial review writes, old review URLs, and company boundaries. - Browser qualification: both GitHub journeys, all five provider setup/management journeys, and the GitHub uncertain-send recovery journey passed. GitHub setup, reconnect, and task replies run with chat disabled; chat enabled alone cannot expose GitHub bots. The disposable qualification database was removed after testing. - Earlier live qualification verified App-owned issue replies, native slash-skill invocation, progress comment edits, and a 5/5 PR review with a passing current-head check in the dedicated permissions-smoke repository. These live checks did not use personal credentials as bot execution authority. ## Risks - Deploy the merged Cloud gateway protocol before the stack update. Each callback and encrypted event must retain the original instance and tenant binding. - Migration 0323 uses idempotent column additions for test installations that already applied the earlier draft migration number. - A verified connection does not prove runtime isolation or a completed review. Runtime checks remain separate. - Required merge checks must be configured in GitHub. A passing numeric score does not formally approve a PR; approval and request-changes tools remain opt-in. - Existing-App credentials cannot silently renew an uncertain manifest exchange. Recovery preserves the existing App, draft, connection, and agent. - Repository changes save immediately. Global toggles affect all currently available repositories, including unloaded rows, without granting new installation access. - A failed or uncertain tool write suppresses automatic fallback text; the governed outbox retains its evidence. Deleted or replaced response comments are not recreated through a personal credential fallback. - GitHub logo upload remains an optional human action in GitHub App settings. Runtime provisioning, delegated setup, screenshot publication, and production push delivery remain deferred. ## Model Used OpenAI Codex, based on GPT-6, with tool use and code execution. The exact runtime model identifier and context size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites; complete test matrix is checked in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 17:01:20 -05:00
DottaandPaperclip	c5f7713dd4	fix: clarify native tool use for external agents (#15734) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - External agents keep their own tools and app connections. > - The Paperclip assignment catalog controls the Paperclip bridge. > - Dot treated a qualification test restriction as a rule for later work. > - Existing prompts did not clearly separate these two tool sources. > - This pull request tells agents to use their own capabilities and return results to Paperclip. > - The benefit is that a missing Paperclip app does not block an available native capability. ## Linked Issues or Issue Description Related work: #15722 and #15691. I searched open PRs for Dot tool guidance and found no duplicate. **What happened?** After a qualification test, Dot refused a Slack request because the test Paperclip agent had no assigned Slack app. Dot already had a working Slack connection in ChatGPT. It carried the test's plugin restriction into a later task message. **Expected behavior** An external agent uses its own available tools, apps, plugins, skills, and capabilities to complete the current request. It then returns the result through Paperclip's task reporting and completion tools. The assignment catalog limits Paperclip calls. It does not disable the agent's native tools. **Steps to reproduce** 1. Connect a Dot that has a native Slack connection. 2. Complete a Paperclip qualification task with a restriction scoped to that test. 3. Send a later task message that requires Slack. 4. Observe that Dot can mistake the Paperclip catalog for its full tool inventory. **Deployment mode** Observed on Cloud staging with the Dot Runner. ## What Changed - Add shared tool-use guidance for external-agent onboarding and Dot setup. - Include it in the server onboarding document, pairing response, event instructions, and capability response. - Clarify the tool descriptions on both Dot bridge paths. - Include the same guidance in each Dot Runner assignment without adding a shared-package dependency to the standalone Runner. - Document native tool use and the scope of test restrictions. ## Verification - The 49 relevant onboarding, Dot invitation, and Runner context tests passed. The final UI run passed 29 tests. The updated plain-text renderer run passed all 4 tests. - Full workspace typecheck, full workspace build, and design token gates passed. All current-head CI gates passed, including full tests, Runner verification, and browser E2E. Greptile is 5/5 on the exact final commit, and the one review finding is resolved. - The real Dot received the standing correction and confirmed that it will use its relevant tools and connections, return results through Paperclip, and stop carrying the test restriction forward. - Extended the existing plain-text renderer test to check native Slack access, qualification-test scope, and the Paperclip denial boundary. This covers agents that receive only the onboarding.txt URL. - The full Vitest suite passed in CI. The earlier local Cloud qualification run encountered failures in unchanged suites; this follow-up uses focused local tests for the instruction changes. ## Risks This changes instructions, not access checks. Native tools retain their own permissions and approval requirements. Paperclip operations remain scoped to the authorized company, agent, and task. Agents must not use another connection or identity to bypass a denial. There is no database migration or new permission grant. ## Model Used OpenAI GPT-6 through Codex. Used reasoning, repository tools, and browser control. The runtime does not expose a more specific model ID or context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 16:20:38 -05:00
DottaandPaperclip	b6b3cb3761	Fix unavailable workspace branches with an inline repair and retry (#15726) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Isolated workspaces must resolve a starting Git ref before an agent starts. > - A task can explicitly select a branch that the repository does not have. > - The current recovery notice calls this a missing secret and offers no branch repair. > - This pull request explains the branch failure and adds a task-scoped repair action. > - Operators can correct the branch and request a guarded retry from the task thread. ## Linked Issues or Issue Description Related: Refs #11892 added unresolved-ref recovery. Refs #15719 added bounded failure diagnostics. This change uses that recovery path and adds its operator repair UI. **What happened?** A task that selects `main` in a repository with only `master` stops before the adapter starts. The task shows a missing-secret message and a generic retry. Repeating the run cannot fix the stored branch setting. **Expected behavior** Explain that the starting ref could not be resolved. Offer a verified repository default, or let the operator enter a branch. Save only the task's starting branch and retry its recorded owner through the existing admission gates. **Steps to reproduce** 1. Create an isolated-workspace task in a repository with `master` and no `main`. 2. Set the task's base ref to `main` and assign an agent. 3. Wait for the configuration failure. 4. Use the inline branch repair and confirm the agent completes in a worktree based on `master`. ## What Changed - Replace the missing-secret diagnosis for unresolved workspace refs. Keep uncertain fetch/access failures described as uncertain. - Read the remote's advertised default branch with the existing Git credentials and a bounded timeout. Do not guess or silently change the branch. - Add an atomic branch repair to the existing recovery resolution endpoint. Check company access, board runtime permission, incident identity, current owner, task/project state, dependencies, interactions, approvals, budget, and active execution before requesting a retry. - Preserve the effective inherited workspace setup when saving the task branch override, and attach one repair notice per failed run even when the incident is reused. - Bind the shared repair card to the exact recovery notice and failed run in both task-thread renderers. Validate branch syntax and prevent duplicate submissions. - Add Storybook examples, regression tests, and a documented local acceptance fixture with a real Git repository and process adapter. ## Verification - Red/green regressions cover the repair endpoint and preservation of inherited agent setup. The regression failed before each fix and passed afterward. - Focused integration suite: 162 tests passed; task-thread/card suite: 230 passed. After review refinements, all 13 repair-route tests, 10 card tests, 77 recovery-action tests, and the repeated-failure heartbeat regression passed locally. - Real browser acceptance passed for suggested-branch and custom-entry repairs. Invalid syntax left the task blocked. Both valid repairs ran the authenticated worker in a real isolated worktree and reached Done; reload preserved the result. - Repeated failures reused the recovery incident and retained an actionable notice on the latest run. The inherited-setup journey verified the provisioned file, agent branch template, worktree parent directory, and actual Git commit before completing. [Before/after screenshots](https://github.com/paperclipai/paperclip/pull/15726#issuecomment-6088734666). - Full local typecheck and build passed; server typecheck passed again after review fixes. Token gates passed. - The monolithic local Vitest run was stopped after resource-contention timeouts and stale assertions. Targeted regressions were rerun; the full CI matrix passed on final revision `26c543ff490bec29ddc77a49ee532c7f84fb96dc`. One signoff E2E worker-cancellation failure passed on a rerun without code changes. The two opt-in Storybook workflow jobs were skipped; the Storybook journey was checked manually. - Repeat the live journey with `tests/e2e/workspace-base-ref/README.md`. Storybook examples use simulated callbacks and are not the live acceptance proof. ## Risks - Default-branch discovery can add up to 10 seconds to an unresolved-ref failure. Lookup failure leaves manual entry available. - A valid branch name can still be absent or inaccessible. The retried workspace preparation remains authoritative and can block again with new evidence. - Retry remains subject to normal admission gates. The repair changes only the task override; it does not alter project or agent defaults. - No database migration or new dependency. ## Model Used OpenAI GPT-6 through Codex, with tool execution, repository analysis, and browser testing. The exact serving variant and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 15:56:14 -05:00
Devin FoleyandPaperclip	98e3cec78e	feat(costs): report monthly subscriptions beside API spend (#15634) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its Costs page reports API charges and subscription token usage. > - Subscription runs do not show the recurring fee paid for the account. > - Several agents can share one paid seat, so a fee per agent would overstate cost. > - This pull request identifies connected subscription accounts and reports each monthly fee once. > - Users can compare API spend and subscription costs, see uncertainty, and enter their actual price. ## Linked Issues or Issue Description Refs #498 and #3406 for related subscription reporting needs. This change does not add subscription budget enforcement or automatic optimization. Related prior approaches: #10544 and #11104 proposed API-equivalent token costs for subscription work. This PR reports recurring account fees separately from usage charges and budgets. A search found no open PR for monthly account fee reporting. **Subsystem affected** Costs, managed AI connections, shared contracts, database schema, and board UI. **Problem or motivation** An organization can pay for subscriptions even when included runs record no additional dollar charge. The Costs page cannot show that monthly commitment or distinguish a shared paid seat from several separate seats. **Proposed solution** Discover account identity from the managed connection selected for a run. Use exact known plans for labeled monthly list-price estimates. Keep ambiguous plans and unconfirmed identities unpriced until the user supplies a price. Sum current monthly fees separately from date-filtered API charges and token usage. **Alternatives considered** A fee per agent double-counts shared accounts. Token-based API price estimates do not measure the recurring subscription fee. Provider invoices would be more exact but are not available through these account credentials. **Roadmap alignment** Extends the shipped Better Budgeting and Personal & Shared AI Accounts capabilities. The feature and UI were reviewed with the maintainer before this PR. ## What Changed - Add company-scoped subscription accounts, connection bindings, and immutable price revisions in replay-safe migration 0322. - Identify OpenAI workspace/seat and Claude organization/account pairs only after provider authentication. Keep provisional records grant-scoped; unchecked ID-token claims cannot combine accounts or grant billing editors. Keep credentials and raw provider identities out of reports. - Attach the selected subscription to new run receipts. Resolve duplicate aliases without rewriting old monetary receipts. - Add cached background discovery with four provider checks and twenty discovery requests at a time. Accepted requests wait for capacity; excess requests receive HTTP 429 and a UI retry action. Preserve saved prices and observations when a provider check fails. Dashboard polls load current prices only; audit revisions stay in the database. - Support owner-controlled monthly or annual prices, fractional-cent precision, separate currency totals, tracking status, and duplicate linking. Preserve both personal owners and shared-account managers when discovery finds the same paid seat. - Show API spend beside monthly subscription commitment, with separate token totals and clear estimate labels. Account details and fee totals follow billing-owner/shared-audience visibility; company-wide token totals remain available to cost readers. - Use consistent amount and estimate badge formatting for agent, user, project, and model rows. - Add real PostgreSQL tests, route authorization tests, UI tests, interactive Storybook fixtures, and operator documentation. ## Verification - Current head `d7b5c78dadf5033a3c447d83434676ac94687e77` passed all 53 checks, with two checks skipped. This includes the full test matrix, browser tests, build, typecheck, migration policy, canary packaging, and security checks. - Greptile Apex reviewed that exact head at 5/5 with no unresolved findings. - Local `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, migration safety checks, and `git diff --check` passed. - All 238 focused tests passed across accounting, subscription persistence, authorization, UI, migration replay, and snapshot drift. - The expanded local chat suite passed 1,149 tests. Two cases failed because this workspace found an unrelated ancestor skill directory; both passed unchanged in an isolated checkout. The full local Vitest command was not repeated after the rebase because of this environment issue. CI covered the complete test matrix on the final head. - Migration `0322_reflective_kree.sql` follows master's `0321_acoustic_human_fly.sql`. Its snapshot and journal were regenerated. Master's migration and snapshot are unchanged. The subscription SQL retains its original content and hash. - Disposable PostgreSQL upgrade fixtures applied master's added lifecycle and Slack migrations without changing saved subscription prices or cost receipts. The renamed subscription migration was recognized by hash, retained one history row, and a second migration run was a no-op. - Regression tests cover provider capacity, fractional-cent edits, billing visibility after grant revocation, and fixed-price/credit/unknown usage. The new UI billing-label tests fail against the old wording and pass with the fix. - Browser checks passed for desktop and mobile layouts, shared-account totals, annual price edits, unknown prices, and stale provider data. - A live OpenAI account probe returned HTTP 200 and a verified Pro account identity. The generic Pro plan correctly remains unpriced. - Live Claude discovery identified Max 20x and a $200/month estimate. Two real Claude agent runs were attributed to that same account, with two usage receipts, two agents, one monthly fee, and zero additional usage charges. Two OpenAI connections also deduplicated to one account. - Live Codex execution is not verified: both Paperclip's managed hello test and a direct host `codex exec` fail with a workspace-routing authentication error. The current credential passes the usage API probe. This is separate from the successful account-discovery check. - A local performance fixture with 50,000 usage events and 5,001 saved price revisions produced a 1,589-byte report in 11.5 ms median / 12.2 ms maximum over ten warmed runs. PostgreSQL used the subscription/revision index to read one current price row. These are local measurements, not a production latency guarantee. - [Desktop, account details, and mobile screenshots](https://pages.paperclip.ing/pr-15634-subscription-costs-d5d285e2/) show the production Costs page with synthetic Storybook fixtures. - To inspect the UI, run `pnpm storybook` and open **Costs / Subscriptions**. Edits there change fixture data only. ## Risks - Provider plan identifiers can change. Unknown plans remain unpriced. Automatic prices are estimates before tax; discounts and negotiated prices require a user-supplied amount. - Subscription fees describe the current monthly commitment. They do not follow the selected usage date range and do not consume agent budgets or create finance events. - Monthly fees cover only accounts visible to the viewer; company-wide token totals retain their existing scope. - Legacy or unmanaged usage can remain unattributed. Unconfirmed identities require manual prices and may need explicit duplicate linking after credential changes. - Disconnecting a connection does not prove provider billing stopped. Users must end or exclude tracking explicitly. - Migration 0322 adds tables, a nullable cost-event column, and an index. It is replay-safe, but normal migration locks still apply. ## Model Used OpenAI Codex, based on GPT-6. This session did not expose a more specific backend model ID or context-window size. Capabilities used: code editing, shell execution, database tests, and browser verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 13:38:01 -07:00
Nicky LeachandPaperclip	2dd9811e8b	Own agent hiring and termination in a lifecycle module (#15631) ... ## Thinking Path > - Paperclip manages AI agents and their work. > - Agent creation, approval, pause, and termination change whether an agent can work. > - These changes currently use separate service and database paths. > - Some agents need setup after their record exists and before their harness can run. > - This PR adds one module that owns those state changes and their transactions. > - The module records progress and retries incomplete work after a server restart. ## Linked Issues or Issue Description **Problem or motivation** A hire can become active before its execution environment is ready. Direct agent state writes also bypass one common lifecycle boundary. Termination does not have a durable step for resource cleanup. **Proposed solution** Use agent lifecycle commands for hiring, approval, pause, resume, and termination. Keep proposed hires pending until approval. Prepare each approved hire, test its saved harness configuration, and then permit work. Wait for required plugin work before completing a step. **Alternatives considered** A general database commit hook would expose transaction details to callers. A timer alone would delay each new hire. This module owns each command transaction and starts its worker after commit. A periodic scan recovers work after a process stops. **Roadmap alignment** This change adds the lifecycle boundary for the existing agent and plugin systems. It does not add a user interface or a provider implementation. It replaces the lifecycle delivery approach in #15391. Related proposals include #11475 and #13099; this change does not add their external executor protocols. ## What Changed - Add durable approval, preparation, verification, pause, resume, stop, and cleanup states. - Keep agent creation, lifecycle state, configuration writes, and agent record deletion behind `modules/agent-lifecycle`. Keep validation, credential rules, reads, onboarding, invitations, general approvals, and company data deletion in their services. - Commit a hire before the remaining onboarding or invitation work. Serialize requests with the existing dedicated database connection helper. Reuse the hire after a failed attempt. - Keep accepted work in the run queue during setup. Require execution and cleanup evidence before completing pause or termination. - Retain the original credential owner when approving migrated hires. - Start lifecycle work after each command commits. Use database leases and revision checks for retries and late results. - Add a generic required plugin method and capability. Store the required plugin IDs for each agent. Track host and plugin completion separately. - Keep harness verification and credential rules in services. Supply their results to lifecycle through explicit ports. Reject failed logins and incomplete probes. - Remove reverse imports from lifecycle adapters into services. Share record queries and validation below both layers. - Use one hire decision path for agent IDs and approval IDs. Resolve the approval and agent change in one transaction. Expose policy reconciliation on the command object. - Save configuration and verification invalidation through one module operation. Ordinary commands own their transaction. Restrict the shared transaction integration to its listed callers. Reject identity, company, accounting, timestamp, and unknown fields. - Require completed termination before deleting agent records. The company deletion service calls the module through a separate deletion interface. - Save an approved secret before applying its agent binding. A failed binding leaves the secret approved. A retry reuses it. - Remove old permission and metadata export files. Move hire configuration rules out of lifecycle persistence. - Report failed worker steps and scans with operation context and known error codes. Omit provider messages, stacks, and configuration values. Document ordinary application use and extension rules. - Recover expired native owners only after execution stops. Retry cleanup of failed retained leases. Wait for setup in the live smoke script. - Keep manual, budget, and company pause reasons separate. Guard execution status writes with the ready state. - Add a migration, boundary checks, regression tests, and a lifecycle guide. Keep the current UI. - Limit periodic policy checks to 100 eligible agents per minute. Continue from the last agent ID. Exclude pending hires and agents in termination or a final state. Keep explicit company and budget updates immediate. ## Verification - Latest head `ccc3c86b0` aligns the shared status update schema with the PATCH route. All 15 shared tests, 104 server route/OpenAPI tests, 2 CLI tests, and shared/server/CLI TypeScript checks pass. The Greptile comment is fixed. CI is running; no new Apex review was requested for this small fix. - Rebase `75708ac94` rebases onto master at `2eb5187e4`. The backup test uses the shared 90-second timeout from master; the older timeout commit was removed. The affected backup test passes. - Agent budget edits now pass the target agent ID through enforcement. All 50 affected tests, server TypeScript, and 11 boundary tests pass. Both configuration update paths have regression coverage. CI is running on the latest head. No new Apex review was requested for these small fixes. - Rebase `9b568666a` preserves the run-control extraction and delivery-work startup changes from master. All 559 affected tests, server TypeScript, and 11 boundary tests pass. CI and Apex review are running on the final head. - The policy scan fix passes 56 lifecycle and diagnostic tests, 24 policy and caller tests, and 11 boundary tests. A regression test checks the page limit, interval, excluded states, and cursor wraparound. - Rebase `fd4698750` passes all 611 tests in eight affected lifecycle, route, and heartbeat suites. Server TypeScript and all 11 module boundary tests pass. The company deletion rollback and concurrent hire tests are included. - Follow-up `00aac7646` fixes the failed decision deletion test and the public update schema. All 126 affected server tests, 8 shared tests, 2 CLI tests, and 11 boundary tests pass. Shared, server, CLI, and UI TypeScript checks pass. - Server TypeScript passes: `pnpm --filter @paperclipai/server exec tsc --noEmit`. - All 11 boundary tests pass: `node --test scripts/check-module-boundaries.test.mjs`. The repository boundary scan also passes. - All 68 lifecycle, failure diagnostic, and harness verification tests pass. Tests reject forbidden fields through the root command, combined transition command, and restricted transaction integration. They also check invalid fields returned by configuration preparation. - Failure tests cover host work, plugin work, claims, renewals, deferrals, and scans. Log tests check known error codes and exclusion of private error contents. - All 355 tests in 10 existing caller suites pass. These tests cover secret approval, pairing, pending hires, credentials, company import, built-in agents, managed agents, instructions, and skills. - The branch is rebased onto `origin/master` at `2eb5187e4`. The queue guards and cancellation lock fix use the extracted heartbeat queue. Agent cancellation uses the extracted run-control file. The source diff scan found no secrets or private URLs. - Earlier checks cover approval retries, rollback, late verification results, policy holds, deletion rollback, concurrent company deletion, and queue admission during setup. - Before the rebase, CI passed on `00aac7646` and Greptile Apex rated it 5/5 with no open comments. CI and a new Apex review are pending for the rebased commit. The branch has no merge conflicts. - The full local build and native Dot suite were not run. The host has no Cargo. No live provider test was run. ## Risks - Background policy recovery takes more than one minute when more than 100 agents are eligible. Explicit company and budget updates still reconcile immediately. - The restricted configuration integration keeps an agent reference and its dependent records in one transaction. It cannot change a budget or invoke a lifecycle transition. A periodic scan starts pending setup after that transaction commits. - Secret approval and binding use separate transactions. A failed binding can leave an approved secret with no agent binding. - New approved hires remain unavailable until setup and verification complete. Existing clients see `paused` during these steps. This PR does not change the UI. - Migration `0320_daily_onslaught.sql` maps existing statuses and pause reasons. It does not create external resources or claim that an existing harness passed verification. - Use a coordinated server update for this migration. Older server versions can write legacy status without the new lifecycle checks. - A failed onboarding or invitation request can leave a committed hire. A retry reuses that hire and completes the remaining records. - Deletion requires completed termination. Company deletion can require another request while cleanup is pending. - Required plugins must make repeated calls safe and reject older revisions. A disabled or removed required plugin blocks completion until it is restored. - A new hire can run a second harness test after the existing UI test. Some adapters can charge for this test. - No live provider test was run. Provider implementation and resource backfill remain separate work. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code tools, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in this PR with the feature request field labels - [x] I have not referenced internal or instance-local issues or links - [x] My branch name describes the change - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] Targeted local tests pass - [x] All Paperclip CI gates are green on the new commit - [x] Greptile Apex is 5/5 with no open comments on the new commit - [x] I will address review comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 09:05:12 -07:00